"the choicest relic”

If you follow the whole quote you will see that Thoreau is enraptured with the beauty of the written word. By relic he means that it is the most valued historical treasure because it represents precious ideas,

"A written word is the choicest of relics. It is something at once more intimate with us and more universal than any other work of art. It is the work of art nearest to life itself. It may be translated into every language, and not only be read but actually breathed from all human lips; -- not be represented on canvas or in marble only, but be carved out of the breath of life itself.”