Robert Klum faced a Seminole County judge Thursday on two counts of child neglect.
Investigators said that at about 8 p.m. Monday, Klum and his two children stopped at a house on Diane Circle in Casselberry.
The homeowner, who asked to not be identified, said Klum had done a few odd jobs for her in the past.
"He was consoling the little boy," she said. "He was consoling the little girl."
The two talked and Klum left.
"I assumed that he had taken the kids home to where he had a place to stay," she said.
An arrest report said the children's mother had been arrested on shoplifting charges shortly after 8 p.m. and Klum was trying to find bail money, but he was arrested shortly after midnight on unrelated charges.
The homeowner said she left her home at about 6:30 a.m. Tuesday and did not see the children.
She said about an hour after she returned home that evening, police officers knocked on her front door, saying that the children had been left in her front yard in a chair covered by a blanket.
"My question is why wouldn't he bang on the door and say call the police to pick these kids up," she said.
Investigators said when they found the children, they were shivering and had been outside for more than 10 hours.
"Anything could have happened to them," the woman said.
It's unknown who called police about the children allegedly being left outside.
Klum is being held in lieu of a more than $16,000 bail.