"I'm getting nothing but negative attention," he said.
But the yelling is not what some neighbors say bothers them the most. It's the controversial signs in his yard.
"This is out of control. This has been going on for so long and we are sick of it. The whole community is absolutely sick of this," said Julian Sands, who works nearby.
Avery has dozens of signs, some with religious references.
"Stuff that is sexist or homophobic or racist or inciting violence, that I think should not be allowed," Sands said.
There's a stuffed animal on display with a toy gun pointed at its neck. Some told WSB-TV they find that and these three stuffed monkeys hanging from nooses deeply upsetting.
"Me, personally, being a black male, it shows a sign that he's racist and not to mention no telling what types of things he has on his mind," another resident said. "To me it's about lynching. It's not only offensive, it's dangerous."
Avery says the hanging "see no evil, speak no evil, hear no evil" stuffed animals mean something else entirely.
"They generally mean being of sound mind, body and action. Set the example for others to follow," he told WSB-TV.
Avery had to go to court for too many signs in his yard and is now on probation.
DeKalb County police say the messaging falls under freedom of speech.