is the poem a piece of advice, given by an elderly person to a young man or woman?

I think on the surface it is the advice given to a woman, "But one man loved the pilgrim soul in you, And loved the sorrows of your changing face". The idea being that there was a man who loved the woman for her soul and the person no matter how age changed her "face". I, like the poem. It makes one think about the what might be when they are "old and grey". One can only hope that they can recall somebody who loved their soul rather than passing physical attraction.