An yet again, it's is still the same thing you mentioned at the OP.
By rare chances you get phobias, but these are TOTALLY random.
In most cases you'll get punished for having bad luck (yeah life sucks, hurr durr, but at some point the game has to go gameplay > realism) .
Not to mention that in the normal game mode the survivor is simply going to save his stuff and charge naked into the horde for a happy all-cleaning suicide.
Oh no, gameplay>realism, my only weakness!
It's not the only one, but moreover the one that would - among others - hit your idea the hardest.
... thinking about what others already mentioned:
Wouldn't "realism>gameplay" also punch you in the face?
We don't know how long it was since the outbreak and yet "we" (the players) survived.
How would that be possible with phobias that would actually hinder you in combat?
And regarding that "you will get some phobia instantly", ScottKrieger for example already mentioned that you won't suddenly get a Phobia.
(While TheArhive is out cold after a Shining Wizard from Gameply, Realism finishes him with a Piledriver!!!)