What reason does the father give for not inviting the little boy they see to go with them?

THe Father can barely feed and protect his own son. Father also trusts nobody. The boy may have adults who seek to do them harm.