why was the transmittal of the memory of snow so exhausting to the old man ?

The memory of snow goes back many generations. The longer away the memory, the harder it is to process. The community is so used to sameness that the memory of snow is complex and exausting to the Giver.

"It's a very distant memory. That's why it was so exhausting - I had to tug it forward from many generations back" (CH.11).