Why Caesar feels he cannot trust Cassius?

Caesar has noted that Cassius is restless and easily covets what is not his. Caesar begins by saying that Cassius is too skinny and "thinks too much"and later claims that he does not enjoy anything of substance like art or music. He adds Cassius keeps alone and reads too much. 

He reads much,
He is a great observer, and he looks
Quite through the deeds of men. He loves no plays,
As thou dost, Antony; he hears no music;
Seldom he smiles, and smiles in such a sort
As if he mock'd himself, and scorn'd his spirit
That could be moved to smile at any thing.
Such men as he be never at heart's ease
Whiles they behold a greater than themselves,
And therefore are they very dangerous