How do we know that Atticus was relieved after the men left?

We know Atticus is relieved based upon his body language and reaction.

I turned to Atticus, but Atticus had gone to the jail and was leaning against it with his face to the wall. I went to him and pulled his sleeve. “Can we go home now?”He nodded, produced his handkerchief, gave his face a going-over and blew his nose violently.