Smelly Ferry
The opening stanza situates a ferry that smell like a horse as the means of public transportation enjoyed by the characters. This indicates that the characters are not people of wealth. They are not so wealthy as to afford private transport, but not so poor as to be unable to enjoy a day of leisure. The ferry symbolizes the economic status of the characters as being distinctly in the middle.
"We"
The first word of each stanza is "We." Neither the speaker nor the companion are identified in any specific way. Even personal pronouns identifying gender are abandoned. The poem constantly presents two people as a single entity. This symbolizes the universality of the events which places them quite distinctly within the broad swath of the average American.
The Moon
The couple enjoys the simple pleasure of lying on a hill and gazing up at the moon. The moon symbolizes the lack of any necessity to pay for pleasure or entertainment for two people who simply enjoy being together. On a deeper level, the moon—which can be viewed by hundreds of millions at any one time—represents a complete rejection of consumer leisure capitalism.
The Morning Paper
The couple buy a newspaper from an old woman which remains unread. The unread newspaper becomes a symbol of the couple's rejection of the external world. They do not care about anyone else's opinions or prejudices.
The Old Woman
The couple address the old woman from whom they buy the paper as "mother." She is not literally the mother of either, of course. They do not only give her money for a paper they won't read, but also give her their surplus of fruit. Ultimately, they give her all their money except for what is needed to pay for the subway. The old woman is another symbolic rejection of capitalism by virtue of being paid in excess for what is not needed or even wanted. She becomes a symbolic incarnation of socialism as the middle-class couple decided to share the wealth they have.