We meet Berry Hamilton, a black man in the South who works as a butler for a wealthy man after being freed from slavery. The estate's owner is Maurice Oakley, and the work goes along nicely. Berry makes enough to support his family, and he raises his children to believe they can do anything in life. One night they Oakley's are throwing a dinner party and a lot of money is stolen. Berry is blamed for the theft, and when they check his bank account, the money is there. Berry is fired and arrested, sentenced to ten years of hard labor.
We learn what becomes of the family members. Fannie tries to support them somehow, but all around town she is known as the wife of the thief. In the Black community, they are largely rejected because they were perceived as less-than the others because they lived comfortable lives living as servants for white men.
Fannie leaves Berry in jail and moves with her kids to New York City. One of the kids is Joe, who hangs out at the Banner Club. He ends up finding work and a girlfriend pretty quick, but the girlfriend works as an adult entertainer, and the other kid, Kitty, ends up working the same job as her. Fannie is dismayed. Joe continues to work in the club where he becomes seriously addicted to alcohol. When his girlfriend leaves him for being a drunk, he murders her, and is sent to prison.
Kitty finds a suitor named Mr. Gibson and marries him for financial support, but he is abusive to her. Back down South, Maurice receives a letter from an old friend where the friend admits he stole the money. Maurice chooses to leave Berry in prison instead of coming out with the truth, but Mr. Skaggs, a reporter, discovers the truth and publishes the story. Berry is set free. Colonel Saunders, a local Southern gentlemen ends the novel mourning for his fallen comrade, for Maurice has been driven insane by humiliation and shame.
