Sabers only? I don't see any really. Only thing that gives him a chance(and pretty much by that extension every other Non-Force User) is as you say Kurk, Jedi morals. They are less prone to use the Force than a Dark Force User or Sith would in combat, or well in an abusive manner that is.
As far as Legends go anyway, in a straight blade confrontation, at best he could hope for is a stalemate.
Let's look at an example here, Prince Xizor vs Jax Paven. Now Xizor's species are above humans in every possible manner, strength, speed, reflexes and so on. Due to these things, Xizor is able to pull off maneuvers that only Jedi would be capable of doing while wielding a lightsaber(noted by Jax), among his extensive martial arts training.
Yet at best during their fight, even with Jax's Force Usage being hampered it was a stalemate between the two.
If you'd like another example, let's look at Phow Ji. This guy was pretty much a superhuman with masters in Echani, Teras Kasi(the same MA Xizor knows) and a dozen of other martial arts. He wrecked several mercenaries(3 of them armed while he did it barehanded), moved faster than holocams, able to dodge blaster bolts(thanks to Echani's precog) and a bunch of other things.
Yet it's noted that he would only be a challenge to Jedi who unaided themselves with The Force.
If you'd like another example, let's look at Darca Nyl, the Mercenary with a lightsaber. He held at bay Lyn's apprentice Sardoth in a duel, yet this was all he could hope fore despite the fact he had been using just a lightsaber for years in combat and nothing else, unlike Vizsla who uses more than just his Darksaber.
So at best in a saber only duel, with no holding back for the Jedi/Sith Force wise? He'll at best get a stalemate.
Canon, well I have no idea on that.