Beauteous Heap (Symbol)
The hill of moss, referred to as a "beauteous heap," could symbolize the dead baby's grave, hence why Martha Ray visits and cries "Oh Misery." The grave is hidden out of sight in a remote location, thereby suggesting that only she knows of its existence. This distance from the locals implies isolation caused by social stigma, as an unwed mother would be shunned in society.
The moss-hill also symbolizes the body of the infant itself. For example, the "tufts of moss" represent the hair, the shaking movement of the grass suggests the baby's movement, and the reflection in the pond symbolizes the baby's face.
Scarlet Cloak (Symbol)
The scarlet cloak that Martha wears can signify her affair outside of marriage, which resulted in a child. Similarities can be drawn with The Scarlet Letter by Nathaniel Hawthorne (written years later, but also part of Romantic literature), where the color red is a prevalent symbol for a seductress and adulterer. The scarlet may also symbolize blood, such as the blood of the dead child that appears to also cover the moss in drops.
Storm (Symbol)
The weather is a symbol of the despair that Martha is experiencing. Rain, frost, and snow are all cold elements and are juxtaposed with Martha's strong feelings of obvious love and adoration for the child. They also represent the storms of her life—from falling in love to losing her lover and then her child. Martha bravely faces the storms, whereas others try to escape them. She fully embraces her pain and is haunted by it, often becoming one with nature when these storms arise.
The Thorn (Symbol)
The thorn is a symbol of the joy and sadness that one experiences in life simultaneously—especially as one grows older and joins society. Within the context of Romantic literature, the thorn may represent how even in the most beautiful natural setting, ugliness can be found. Even something as beautiful as the moss can exist as a foe in nature, dragging the thornbush down. Beauty has the power to destroy, just as the beauty of Martha's relationship is lost when her lover abandons her. Her beautiful child is also lost, changing from a source of joy to one of utter pain. The thorn may also symbolize grief and affliction. Growing up is a kind of affliction of the human experience, as love, innocence, and truthfulness are lost.