According to Chillingworth, how have he and Hester wronged each other?

Chillingworth claims that he has wronged her by marrying her in the first place. He says that he an old "cold intellectual" has no place marrying a young vibrant girl like Hester. Chillingworth also thinks that, by not divulging the name of Pearl's father, she has wronged him.

Chillingsworth has wronged Hester simply by marrying her. When they were married he was much older than her and, quite simply, dull. Whereas she was vibrant, young, and full of life. In this way Chillingsworth feels as though he stole Hester's youth away. Upon confronting her in the prison cell, under the pretense of providing medical assistance, he admits to her that he, a studious bookworm, really couldn't have expected to keep such a beautiful wife happy and so he understands that she has had an affair, but still believes that she has wronged him for it and for not telling him who her lover is (was).