By Tom Kludt

NEW YORK (CNNMoney) -- A weekend-long backlash over a decision to invite alt-right provocateur Milo Yiannopoulos to the Conservative Political Action Conference ended Monday with an abrupt reversal.

Yiannopoulos, an editor at Breitbart News, will no longer speak at the annual confab, a decision that appeared to catch him off guard.

"I haven't heard any indication that they are reconsidering," Yiannopoulos told CNNMoney early Monday afternoon.

That changed less than an hour later, with the sponsor of the conference, the American Conservative Union, announcing that it had rescinded Yiannopoulos' invitation.

The reason for the turnabout: a pair of video clips that surfaced Sunday in which Yiannopoulos appears to be speaking sympathetically about pedophilia and cracking a joke about his own sexual encounter with a Catholic priest as a child.

"We continue to believe that CPAC is a constructive forum for controversies and disagreements among conservatives, however there is no disagreement among our attendees on the evils of sexual abuse of children," ACU president Matt Schlapp said in a statement.

President Trump is scheduled to speak at the conference on Friday.

TM & © 2017 Cable News Network, Inc., a Time Warner Company. All rights reserved.