Astro, quick clarification question for you. Is the matter gone or cannot escape? Do we know?
By the way, far more eloquent explanation than mine. I think you would love these lectures.
Professor Katherine Blundell was so inspiring; she nearly had us in tears. The video is good, but the good Professor painted such a vivid picture with her Cockney accent and enthusiasm, and out of this world knowledge. It was almost like she led us by the hand through this adventure.
The matter isn't "gone" - it still has mass, which is added to the mass of the singularity, but it is no longer interacting with the rest of the universe outside the event horizon. Whatever that matter was - it is not that anymore - most likely. It's hard to know exactly what is going on inside - but since we cannot know - the question simply becomes pointless.
When a black hole "evaporates" via Hawking radiation - there are new particles formed just outside the event horizon. Because they are new, nothing of the matter that was accreted by the singularity will be returned. Not the form, shape, or function. That matter is basically gone.
Hawking radiation, by the way, is a brilliant deduction by Stephen Hawking. Quantum mechanics predicts that particles and their anti-particle (for example, electron and positron) can randomly form and annihilate each other, without violation of energy conservation, or any other principle of thermodynamics. So Hawking's brilliance was to ponder - what if they formed right on the edge of of a black hole, and one fell in, while the other escaped? So, the escaping particle would now be subtracting from the mass of the black hole and particle by particle, a black hole could evaporate over time, if it were not adding more matter.
This is the likely fate of primordial black holes, which are likely tiny. Only from perturbations in the formation of the universe would we get tiny black holes, because the rest are formed either by the death of a star in a supernova, or by the hundreds of millions of stars in the center of a galaxy, and as a result, are pretty big.