Well...I don't think that 20 years without fighting would do anything to your lightsaber skill.
We've seen some people in the movies who were damn good at lightsaber combat without having that much fighting experience. Sidious didn't probably have a fight from TPM to ROTS times dispite that he could still kill 3 Jedi Masters. Dooku didn't have much practice between TPM and AOTC either and he still could play with Anakin and Obi-Wan and survive Yoda.
Now Vader (from Mustafar on):
a) he lost 60 % of his raw power according to Lucas
b) he lost both legs on Mustafar (so he won't be too fast)
And now imagine Ben as something like Dooku or at least ROTS Obi-Wan with 20 years more experience with the force. He would kill Vader.