In Chapter 19, why does Garvey say they are all going to head back home?

From the text:

“I’m dog tired, and I’m not your slave. There’s cold cereal over there if you want some.” Cole crawled into the green canvas tent.

“Have a good sleep,” called Garvey. “Tomorrow, we tear down the shelter and head back to Minneapolis.”

Cole poked his head back out the tent flap. “What are you talking about?”

“You’re finished here,” said Garvey, his voice hard and absolute. “There’s not enough room on this island for both you and your attitude.”