Please reference what the text says sin and crime are.

There is no deffinition that I can recall but sin and crime are mentioned throughout the story,

If thou feelest it to be for thy soul's peace, and that thy earthly punishment will thereby be made more effectual to salvation, I charge thee to speak out the name of thy fellow-sinner and fellow-sufferer!" Ch 3

Thus the young and pure would be taught to look at her, with the scarlet letter flaming on her breast,—at her, the child of honorable parents,—at her, the mother of a babe, that would hereafter be a woman, —at her, who had once been innocent, —as the figure, the body, the reality of sin. ch 5