I mentioned this in my post about undead, but I will go ahead and repeat it here. I dislike level draining undead. These creatures, especially in older editions, are brutally unbalanced. The reward you get from defeating something like a wraith or a specter doesn’t come close to the risk of your super cool guy you’ve been playing for months or years getting busted down a few levels. Does anyone ever want to keep playing their eighth level fighter if in the space of one combat he’s a third level guy again? No, they don’t.
As a Dungeon Master, you have three options when it comes to using this type of creature. The first option is to let the dice fall as they may, possibly blasting your party all the way back to first level. The second option is to just have it miss all the time, or possibly fudge dice rolls to make sure that it only knocks a level or two off a couple of guys. The third option, which is often tacked on to one of the above options, is to make restoration spells and scrolls readily available to mitigate the damage done.
With the first option, you’re telling your party that you’ve got some really great first level adventure ideas that you never got the chance to try out the first time around with these characters. The second option is probably going to result in a couple of players rolling up new characters depending on how hard they get hit. Finally, the third option makes the whole fight pointless; if you’re going to give the party something that makes it like the fight never happened then why not just make the fight never happen? I guess a fourth option would be to make the party pay out the nose for all the restoration magic they need, which could be a dirty way to get rid of some magic items you wish you had never given them in the first place.
I realize there are some other dangerous creatures that just make you roll a saving throw or die, and I’m more ok with that (although I tend to use such things sparingly). I’d rather kill a guy outright than force him out of the party because he can’t hang out with his friends anymore (“You guys go on ahead to White Plume Mountain, I’m gonna head back to the tavern and see if I can find someone who wants to find out what that Sinister Secret in Salt Marsh is that I’ve heard so much about…see ya around!”).
I’m sure a lot of people would argue that these guys are so terrifying to fight because of what they can do to a party; I’ve just never wanted to deal with what they can do to a party by running low-level adventures again, forcing characters out of play, or using DM fiat to make it so that nothing bad ever happened. So, in summation: screw specters.