Since WW2, over the years Fury started losing sight in his eye. Its not like he just got the handicap. If he and Punisher were to fight the week after Fury completely losing his eye, then Punisher would indeed have something to exploit. Its not the case now and Punisher would know that. Considering he knows about Fury. If he didnt know him Punisher says to himself....
"This one-eyed joke thinks he can stand in my way?" "An easy victory for me. Yesss!"
Then gets his ass handed to him.