Where, in the novel, does it say anything about Hester's isolation/being outcasted within the town?

There are quite a few points in the book that deal with Hester's isolation. Consider Hester put on display while the whole town could gaze and mock her,

Measured by the prisoner's experience, however, it might reckoned a journey of some length; for, haughty as her demeanor was, she perchance underwent an agony from every footstep of those that thronged to see her, as if her heart had been flung in the street for them all to spurn and trample upon. Chapter 2