Why did Anton and Ruth feel that it was time for Anton to leave his hiding place?why didn't patty agree?

Anton had come to Ruth's aid when he father beat her, and in turn, she told Patty the truth about Anton and who he was. When Patty cooks breakfast for the three of them, a car pulls into the drive, and there's a rush to hide Anton before anyone notices his presence. It's at this point they believe it is best for them to leave. Patty doesn't agree because she feels he is safe with her.