what are the reasons the narrator and maxim shouldn't have come to manderley?

Well, probably because Manderley was a place full of bad memories and "skeletons in the closet" so to speak. There was much that Maxim did not tell his new wife about his past. In particular he did not reveal the nature of his relationship with his first wife Rebecca. He also didn't tell her about his creepy housemaid who keeps a shrine to Rebecca in a room. Basically everything from Maxim's past comes "floating" back to the estate including the shipwreck that Rebecca’s body was stuffed in.