Beyond Hester’s explanation, why won’t Pearl come to Hester without the scarlet letter? In chapter 19.

Pearl has never known her mother without the letter. It's as if it is a part of her. When Hester casts the letter away, it might easily be inferred that Pearl thinks it possible she might be cast away as well.