But Phillips told "Today" host Savannah Guthrie on Thursday that he did hear the students shout the phrase.
He added that he was "upset" after watching Sandmann's interview and reading his earlier statement.
"Coached and written up for him – insincerity, lack of responsibility – those are the words I came up with," Phillips told Guthrie. "But then I went to go pray about it, and then I woke up ... with this forgiving heart, so I forgive him."
>> Read more trending news
When asked if he wishes he had just walked away, Phillips said: "That's what I was trying to do."
Phillips also said he, like Sandmann, has received death threats in wake of the incident.
"You know, I didn't have any problems until the students started saying that they were getting death threats, and then as soon as that happened, it started happening with me," Phillips said.
Read more here.