What is the reasonbehind the narrator's conflict with the Brotherhood? In Invisible Man

The reason the members of the Brotherhood are so angry is the nature of Clifton's death, as he was selling such a degrading object when he died. They call Clifton a traitor to which the narrator responds that the shooting of an unarmed man is more important than obscene dolls and that the new program they had been using has lost much support.

They are appalled that he thinks he has the authority to claim personal responsibility and one member points out that he has become a danger to the Brotherhood because of his claim. His purpose to the Brotherhood is finally spelled out explicitly when Jack tells him he was not hired to think. The committee must strike back and try to silence him. As their puppet, he is not meant to think.