why do you think neighbour does not want to pull down the wall?

The neighbour has but one answer to the question-

He only says, "Good fences make good neighbors."

The implication is that the neigbour appreciates the boundary between them and wishes to keep the distinction clear. The narrator sees this as pointless as the fields are not livestock areas-

He is all pine and I am apple-orchard.

My apple trees will never get across

And eat the cones under his pines, I tell him.

The neigbour enjoys the cameraderie of maintaining the wall, but wishes to maintain his privacy. It is obvious that the neighbour has continued his family adherence to this view-

He will not go behind his father's saying,

The narrator, however, sees this view as old fashioned-

He moves in darkness as it seems to me,

The neighbour himself does not seem to understand the point of the wall. The repetition of "good fences make good neighbours" shows his uncertainty in anything but his father's beliefs.

Frost's description of his neighbour as an "old-stone savage" reinfoces this idea of the neighbour's ignorance in his old fashionedness.

"He will not go behind his father's saying" suggests his fear of betraying his father's tradition (possibly honouring his memory?) or his lack of understanding and guidance in the field.

maybe the neighbour recognises that the wall is what brings these two men together? or that the wall gives them both the security and sense of identity the need in order to remain happy and peaceful?