scout states that there is a “castle system” in Maycomb. How does she explain the system?

I think you mean caste system. A caste system is any system in which people are divided in some way, most often by ethnic, racial, or financial lines. In Maycomb, there is the obvious white upper class, represented by Atticus and his friends and family; there are the poor black citizens represented by the citizens such as Tom Robinson; there are the "other white citizens" who, for example, make up the jury; there are the poor white "trash" represented by Robinson's accuser and her family. These people represent different ways of living and indeed a different place in the social order.