From the text:
He was masked; it could not be said if he was black or white, and therefore proved the property owner was no racist.
There were riots, but these were outside the city, where people of another color were quartered. These people were not allowed into the suburb except as reliable housemaids and gardeners, so there was nothing to fear, the husband told the wife.
The effect of the author never explicitly telling us the race of the family leaves us to make up our own minds..... to go with the picture we naturally build in our minds. In my opinion, the author wants the reader to look within themselves.... to see their own assumptions and preconceived notions without directly pointing them out.