Why does the narrator call the lawyer an unhappy man?

The banker assumes that the lawyer must be miserable to agree to such a wager. He implies that the lawyer must be unhappy if he is going to sacrifice some of the best years of his life over a bet.

"Think better of it, young man, while there is still time. To me two million is a trifle, but you are losing three or four of the best years of your life. I say three or four, because you won't stay longer. Don't forget either, you unhappy man, that voluntary confinement is a great deal harder to bear than compulsory. The thought that you have the right to step out in liberty at any moment will poison your whole existence in prison. I am sorry for you."