If the Captain of the Dutchman destroyed his own heart, I believe that one of three things would happen; Calypso would choose a new captain, the ship would simply vanish, or the ship would sail the seas for forever, its crew trapped between life and death for eternity.
Following movie-verse, I believe the most likely action would be Calypso would create a new Dutchman with a new captain. Of course, I base this only on my own imagination since obviously this subject wasn't really covered in the film. 😉
As for whether or not Davy Jones would have been capable of making the choice to stab his heart, I would say "No". Considering the makeup of his personality, he held onto his "life" with the tenacity all petty, sadistic rulers do. He never desired death; he wanted only his vengeance on the world.
Really, it's very depressing when you think about it.