Why does Pip feel uncomfortable visiting Satis House with Joe?

Having been made to feel "common" by Estella, Pip is certainly reluctant to take Joe to Satis House. But, in Chapter XII of Great Expectations, Miss Havisham has talked with him about his being apprenticed, asking, "Would Gargery come here with you, and bring your indentures, do you think?" So, in Chapter XIII, Pip is dismayed as he sees Joe dressing in his Sunday clothes in preparation for their visit to Miss Havisham because Joe does not look natural; he clearly is uncomfortable with the high collar.

After they arrive, Pip takes Joe to meet Miss Havisham, who is seated at her dressing table. When she addresses him, "You are the husband of this boy?" Joe does not respond. Instead, he talks to Pip in long, awkward sentences thick with his lower-class dialect. As he pulls out the papers of indenture, Joe hands them to Pip, who narrates,

I am afraid I was ashamed of the dear good fellow--I know I was ashamed of him--when I saw that Estella stood at the back of Miss Havisham's chair, and that her eyes laughed mischievously.

Upon their return home, Pip feels ashamed. He says, "It is a most miserable thing to feel ashamed of home." Joe had sanctified this humble home, Pip narrates, but now "all this was changed." Now, it is "coarse and common" because of his having encountered Estella and Miss Havisham.