"And fly with me to Love's Elysium"
The poem is about a man fantasizing of being intimate with a woman and persuading her to experience that intimate bond between man and a woman beyond religious and social rules and judgments. The entire poem is made out of erotic metaphors and this is only one of them to compare this act to something otherworldly, to a state of intense and complete pleasure hence Elysium.
He bids me fight and kill ; or else he brands
With marks of infamy my coward hands.
And yet religion bids from blood-shed fly,
And damns me for that act.
Honor is a personified character who is self-contradicting in the sense that he forces the man to fight and kill or else he will be marked as a coward. However, the religion to which that honor is connected is against blood-shed, so either way the man is in the wrong in whichever way he chooses to live.
No wedlock bonds unwreathe our twisted loves,
We seek no midnight arbour, no dark groves
To hide our kisses : there, the hated name
Of husband, wife, lust, modest, chaste or shame,
Are vain and empty words, whose very sound
Was never heard in the Elysian ground.
The speaker of the poem dreams about an erotic, Utopia place where the act of love isn't frowned upon by religious or social principles, where it isn't something to be ashamed of. This idea of marriage being a tool that diminishes love as it is used to strengthen the bond between the partners is a paradox in itself. The speaker frowns upon these terms and concepts behind them because of their purpose of limiting and controlling the human nature.