Obviously, the joke is a metaphor of some kind, but to what exactly? It might be a picture of the pain of genius-level intelligence, which often makes people feel crazy and isolated, like these admittedly crazy, literally isolated Physicists. But then again, the fact that Mathilde von Sahnd is such a dominating character implies that actually, there is one more element in understanding this puzzling plot. Simply put, the characters are in Oedipal nightmares.
The men often become increasingly dysfunctional because they are isolated, and because their only attention from humans comes from women with whom they are not allowed socially to be romantic with (because they are professionals and nurses; after all, this is a psych ward, which is not exactly a romantic environment). Ironically, the social response (painted symbolically in the Les Cerisiers Sanatorium) is a picture of the very dilemma that left the physicists feeling so frail and extremely volatile in the first place.
What is it? Simply put, it's loneliness. Although the novel's depiction of genius involves compulsory institutionalization, the metaphor applies to everyone who has ever been too smart to fit in. That is a lonely place to be, especially since geniuses are people who are naturally gifted and curious. Those people are often also more sensitive, and the isolation of not being understood properly can quickly spiral, until it feels like this book describes.