Doing that opens the system to being gamed easily. If the penalty is temporary, then a large number of logins can be acquired and scripted to enable down-voting cabals. If the penalty is permanent, then new accounts with sufficient karma would need to be "farmed."
Maybe pg's spam controls are just excellent, but it really doesn't look like HN is being gamed too hard. There are the usual serial self-promoter/spammers (the kind that submit everything they post, regardless of quality), but I really haven't seen that many problems. Perhaps we shouldn't be as worried about flags being abused.
I think we should be worried. From what I've seen on reddit, downvote abuse seems much worse than upvote abuse. Flag abuse would be much the same as downvote abuse.
The power of upvote cabals is often dwarfed by the will of the majority. Downvote cabals can prevent material from being seen by the majority, so they are inherently more powerful. (The distance from 1 to -5 is much less than 1 to 99.)
I can easily see the point of an upvoting cabal -- you'd use it to get your stories onto the front page. And you'd only need five accounts in order to do that quite successfully.
What's the point of a downvoting cabal? To allow you to severely punish comments you don't like? Sounds petty even by internet standards. And if you do manage to get a comment down to -4, then I notice that comments in severely negative territory which aren't obviously awful tend to get rescued anyway.