The Great Gatsby, drifted his way

Could you please tell me the literal and metaphorical meaning of "drifted his way" in this excerpt from chapter five of The Great Gatsby by F. Scott Fitzgerald?

There must have been moments even that afternoon when Daisy tumbled short of his dreams—not through her own fault but because of the colossal vitality of his illusion. It had gone beyond her, beyond everything. He had thrown himself into it with a creative passion, adding to it all the time, decking it out with every bright feather that drifted his way. No amount of fire or freshness can challenge what a man will store up in his ghostly heart.

Thank you.