Exactly. While I think that Heath Ledger's performance was supremely memorable and one of the finest rendered villains in modern cinema, the value of a director in this should not be taken out of the equation. For example... do you think Heath Ledger's performance would've achieved the same level of credit had it not been for the tense and unnerving atmosphere that engulfs the movie every time he shows up? The cinematography, the score (the rising, static-like noise in particular), and the dialogue all help craft his performance.
Of course, this is perfectly acceptable; all great performances were helped by the people behind the camera. But still, it's impossible to forget Nolan's ultimate contribution to the movie. He was the genuine 'star' of the film, in my opinion.
All in all, I'd rather have seen TDK rack up nominations for best picture and director and have Heath snubbed than vice versa, although a perfect world would have both.