what is so tragic about it?

Tragic? Yes, they have a choice, but either choice they make is a gamble. I think that they stayed together makes it less of a tragedy than it would have been had they been separated.

It is a painful irony that Eliezer and his father decide to be evacuated from Buna with the rest of the prisoners. Wiesel's tone in describing this tactical mistake is understated and quietly ironic. After Eliezer suggests to his father that they leave with the other prisoners, his father replies, "Let's hope that we shan't regret it, Eliezer." The very next paragraph reads: "I learned after the war the fate of those who had stayed behind in the hospital. They were quite simply liberated by the Russians two days after the evacuation." This casual, almost off-handed paragraph clearly conveys the bitter regret that Wiesel feels for having made the wrong decision, and like the narrator, we are haunted by how things could have turned out differently had they decided to stay.