The 13.5 billion years of darkness before I was born is frightening as well, but it's done and over with, plus it's way shorter than the upcoming darkness.
The universe itself is the life you have, it doesn't die, just makes more conscious entities. Life itself will not die with me, but go on forever. We just need to think large when contemplating death.