Thor hit him a total of 5 times and he was done....
What challenge did he pose? The close line that thor laughed off? the punches that didn't appear to cause him pain or the choke hold he broke with one hit? Maybe it was the throw that thor got up and WTF pwned him with a single drop kick?
watch the video again and watch how Thor is running through trying to get his hammer. He stops beats up few guys in his path then runs from some other people. He just beat a group of people why run from another especially when he showed he could handle them quite easily? The reason isn't because he is scared it's because he wants his hammer. He gets surprised and smiles fights back and instead of drawing out the fight he ends it with a drop kick. The only person that looked like a threat to me was Hawkeye.
I get your point though. A guy got a few blows in and tbh failed entirely at incapacitating him at all. But your thinking is if this guy got a few hits in what would prevent Bane? My simple answer is...Thor wont be distracted by the thought "I want to grab my hammer and leave here" he will actually be ready to compete in a full on fight. He is a highly trained warrior with TONS of experience that won't take light of Bane.