Why Does Hester Prynne associate Chillingworth with the Black Man?

Black Man

a nickname for the devil. The legend speaks of a Black Man who inhabits the woods and gets people to write their names in his book, using their own blood as ink.

Hester associates Chillingworth with the "black man" because of his influence over Dimmesdale.

Two or three individuals hinted that the man of skill, during his Indian captivity, had enlarged his medical attainments by joining in the incantations of the savage priests, who were universally acknowledged to be powerful enchanters, often performing seemingly miraculous cures by their skill in the black art. A large number--and many of these were persons of such sober sense and practical observation that their opinions would have been valuable in other matters--affirmed that Roger Chillingworth's aspect had undergone a remarkable change while he had dwelt in town, and especially since his abode with Mr. Dimmesdale. At first, his expression had been calm, meditative, scholar-like. Now there was something ugly and evil in his face, which they had not previously noticed, and which grew still the more obvious to sight the oftener they looked upon him. According to the vulgar idea, the fire in his laboratory had been brought from the lower regions, and was fed with infernal fuel; and so, as might be expected, his visage was getting sooty with the smoke.