Why doesn't Gene ever develop a hatred for his enemy? And who (or what) is his enemy?

Gene's only enemy is himself. He lives for years with a guilty conscience and the memories that go with it, and yet he in unable to forgive or to forget. But memories can be colored over time, and everyone's memories are individual. No two people see things in exactly the same way. It's like Gene's memory of the tree, and the trees symbolic meaning of his past;

"It had loomed in my memory as a huge lone spike dominating the riverbank, forbidding as an artillery piece, high as a beanstalk," he says, his similes characterizing the tree as a great, forbidding mass (5).

The tree has a significant meaning for Gene, but his memory and the reality can't coexist when put to the test. When revisited the tree appears "absolutely smaller, shrunken with age," but the tree isn't dead; the memory is just inaccurate.