Why does Jack feel guilty for sitting with Summer and August?

Jack feels guilty because he sees himself as a hypocrite. Initially, he refused to sit with August, now, he really has no one else to sit with, and Summer was kind enough to ask him to join them.