Why do you think Elizabeth would have replaced the veil if it had fallen away?

By that time the people were used to the veil on the priest. They had conjured up their own conjectures about him. At times they ignored their own sins and fixated on the veil. Even worse, they had projected their own hidden sins upon the priest behind his veil.