"It's making him look bad, frankly," Trump said of Lewis in an interview with Fox News.
Other Republicans from Congress joined in that criticism.
And there were even some Democrats who said it would be better for their colleagues to show up at the U.S. Capitol on Friday.
So far, the only Democrats who aren't going to the Trump Inauguration are from the House, as more liberal Senate Democrats have not yet broken ranks to boycott Trump's swearing-in.
The number of Democrats staying home kept growing throughout Tuesday.
Reports indicate that Mr. Trump will only speak for about 20 minutes in his Inaugural Address; he is also slated to have a shorter than usual parade.
Supporters say there's a reason - he wants to get to the Oval Office as fast as possible, to begin undoing many executive actions of the Obama Administration.