Why does Dimmesdale tell Hester that she can be happy while he cannot?

Dimmesdale says that unlike Hester, his shame has ot been publicly announced. Thus, happiness will always allude him because he must suffer in secret.

Happy are you, Hester, that wear the scarlet letter openly upon your bosom! Mine burns in secret! Thou little knowest what a relief it is, after the torment of a seven years' cheat, to look into an eye that recognises me for what I am! Had I one friend—or were it my worst enemy!—to whom, when sickened with the praises of all other men, I could daily betake myself, and be known as the vilest of all sinners, methinks my soul might keep itself alive thereby. Even thus much of truth would save me! But now, it is all falsehood!—all emptiness!—all death!"