"I don't understand why the Senator from Kentucky wants to insist on shutting down the federal government," said Sen. John Cornyn (R-TX), the number two Republican in the Senate.
But Paul stood his ground, leaving his colleagues aggravated, as he demanded a more open budget process, and accusing his fellow Republicans of embracing a huge increase in spending, which he said might lead to a $1 trillion deficit this year.
"If you were against President Obama's deficits, and now you're for the Republican deficits, isn't that the very definition of hypocrisy?" Paul asked on the Senate floor.
Paul ridiculed the budget deal, which will increase defense spending by $165 billion over two years, and add $131 billion in non-defense spending in that same period.
The budget deal also includes $89 billion in hurricane disaster relief, as it totals close to $400 billion in new spending.
Some Republicans were ready to join Paul, frustrated by the details of the agreement.
"This budget deal shows the American people exactly how broken our budget and appropriations process is," said Sen. James Lankford (R-OK).
"It does not address our runaway deficits, and actually takes major steps backwards in the fight to reign in Washington’s overspending appetite," Lankford added.
With Sen. Paul blocking action on a funding plan, Democrats were more than happy to blame the GOP more broadly for the shutdown - no matter how short it lasts.
Senators were expecting a final vote in the Senate by around 3 am; the bill would then be rushed right to the House, for action there by sunrise.
"At this point, we expect next votes in the House to occur at very roughly 3:00-6:00 a.m.," Republicans were told by their leadership.
A number of more conservative GOP lawmakers in the House were ready to vote against the bill, but Republicans were counting on a number of Democrats to vote for the plan, and keep it on track.