what made mr ross realize that he must bring the experiment to an end?

Eventually, teachers students, and even Mr. Ross' wife acknowledge that things have gone too far. Mr. Ross' group is out of control; they indulge in vandalism, terrorism of other students, and one of the students went so far as to appear at Ross' home like it was his own. Ross himself began to enjoy the experiment far too much; he enjoyed the power, the way the students looked at him, and the way they hung on is every word. Once he realized the entire experiment had escalated to a point where he could no longer reign the children in, he knew it had to end.......... it was a fast spreading cancer, and those involved learned a very important lesson about blindly following a leader.

After talking with Laurie and David, as well as his wife, Christy, and the principal (Mr. Owens) , Ross realizes that The Wave has taken a turn for the worse, and he is determined to stop it.