What is the significance of the image of the Satue of Liberty, her torch "almost lost in the fog" in Invisible Man

The statue of Liberty contains the message, "Give me your tired, your poor, your huddled masses yearning to breathe free..." This should apply people like the narrator but somehow the message is lost in the fog; the old vet had told him to step out of the fog.