What does Esperanza mean when she says her neighborhood is brown?(28)

Esperanza means that her whole street is full of Latinos. They are tough because they are used to the status quo where everyone is brown,

All brown all around, we are safe. But watch us drive into a neighborhood of another color and our knees go shakity-shake and our car windows get rolled up tight and our eyes look straight.