Caius's big points are A:He can't die (The spirits of Yeuls bring him back, and he can't even suicide.) And B:His Apocalypse is significantly more destructive then Kefka's. Kefka rearranges the face of the planet. Caius stops time. Forever. Hence, Nova Chrysallia.
If he took a Light of Judgement (or Meteor, for that matter) to the face, the Yeuls would force life back into him whether he wanted it or not.
He also tends to have an edge on strategy due to time travel Shenanigans. "I have seen how it begins and how it ends." On a similar note, he can keep fighting 700 years in a row with nary a rest.
Anyways, Kefka has to figure out how to kill someone who can't die. Or at least permanently disable. If he can't do that, he'll lose to a war of attrition, as Caius keeps coming back every time he's killed.