I believe that Faber's guilt stems from his cowardice. It is very easy to see that he is not brave, such as when Montag has to convince him to go to his printer friend and when he gives Montag the earpiece so that he is with him without actually being there. While he is one of the few intellectuals left in their society, I don't believe that he had much in the way of responsibility because they were fighting a losing battle. Even if he had done more, it ultimately would have done nothing because the minds of the mass would not have been swayed by one more mere voice.
Thus, his guilt stems from his inability to act, but he is being too hard on himself as there is only so much that one person can do, especially facing the odds that he did.