The witch
The witch herself, who looms over the film's events and characters because of her greed and dastardly actions, symbolizes the constraints and burden that Puritanical religion and patriarchy place on women. At the start of the film, Thomasin and her family are bound by the restrictions that their religion places on them. Women tend to the home; men to matters outside of it, including financial and practical concerns, like finding food, with an iron fist. Neither men, nor women are allowed to exist outside those defined roles, though.
The witches exist outside those societal expectations, allowing women to explore different sides of themselves and exist outside the restrictions that society places on them. Witches are boundary breakers; they ignore religion and patriarchy and live the lives they want to -- much like the feminist movements sought to do centuries later.
The goat
The history of goats as a symbol of evil and the devil can be traced back to 4th Century Paganists. That symbolism continues in The Witch, where viewers see Thomasin and her family interact extensively with goats. After Thomasin and the other children upset their father, he locks them in the goat stable in a fit of rage. The witch, the film's antagonist, enters the stable and feasts on the blood of a female goat. Throughout the film, goats are linked to evil and the devil. The witch and Black Phillip (a goat that takes human form) uses the family's goats as a pretense to kidnap Samuel and Thomasin's twin brothers. Goats are in many ways the embodiment of the devil, doing things for their own benefit.
The coven
The witch’s coven symbolizes togetherness and community. For much of her life, Thomasin felt alone and isolated from herself and the rest of the world. She felt stuck in her life, and the way her life needed to be conducted. However, she is eventually led to the witch’s coven, where she finds people who think like she does and have similar interests. In the coven, she finds a place where she can discover more about herself and the world around her.
The raven
In a hallucination, Katherine takes off her clothes and lets a raven gnaw on her skin. In the film, ravens symbolize the death of her soul. As they gnaw on her skin, she gets closer to her spiritual, and later, physical death. Her youngest has died, her family is intensely feuding, and her twins have been abducted. She sees no reason to live any longer, so the ravens unburden her.