The centrality of power
Instead of seeing the world as inherently complex and confusing, Nietzsche says that there is a common thread that can be seen as the central drive of the human life: power. Power is the ability to secure one's desired outcome from a given circumstance. He says that by prioritizing power as the only true currency of the human experience, a person can reorient themselves to this primal truth, to steal back the power that society has taken from them.
Reincarnation
For someone who is admittedly not religious, Nietzsche is more than generous in considering the mystic strangeness of reality and the human experience. By rejecting one religious default, he doesn't arrive at typical atheism—he arrives at existential absurdity and confusion. He admits that it is completely plausible that we are alive to be tortured, and that after death, we would continue being reincarnated into the same life. This idea complicates the issue of life and death with infinite regression, and it can be seen symbolically as an admission of existential powerlessness.
God as dead
The idea that God could die is a commentary not about God, per say, but about human beings. In other words, the fundamental assumption that we should adhere to the religions of the earth can now be seen for its absurd value. The human has become enlightened to the horrible truth that technically, nothing can be known about heaven or the afterlife except by ceding one's power to a religious organization. By accepting responsibility for one's journey to power, one must abandon adherence to belief systems. God is dead in the sense that, to Nietzsche, the considerations of religion are only relevant to cowards. That's how he explains it anyway.
Buddhism as a counterpoint
One might say that Nietzsche's atheism bears strong resemblance to the Buddhist enlightenment. This is an observation that he mentions in this book, noting that, like Buddhism, he has moved past the petty power games of religion and "belief" to a mindset far past belief. He accepts that religion exists, and will probably always exist, and that he is powerless to awaken those who choose to stay in the darkness of "belief," but Nietzsche himself understands that life is horror and sorrow, and that the best we can hope for is escape from reincarnation; this is similar to the Buddhist idea.
The madman and the sun
This is Nietzsche's allegory through which he critiques atheism. Although mostly, The Gay Science is considered to be anti-Christian or anti-religious, Nietzsche clearly aims the thrust of the book at the academic, atheistic society. He despises popular atheism because it still operates as a religion. For instance, in the parable or allegory, we see a madman walking into town carrying a candle, and the atheist community greets him. They ask about his candle and he says that they still believe in the sun. The metaphor is that true transcendence of religion involves absolute skepticism. Not only can one not trust religion; one cannot trust any idea by faith. He says too many atheists still go on living as if life is inherently meaningful, though they profess it is meaningless.
