You're thinking of the public opinion and that's important. But more depth is needed. Essentially, Amon asked them to cancel a football game. The whole "I don't negotiate with terrorists" is fine but it's not like he is picking something like, to use your example, a bending hospital. Why does he pick something small though, in the grand scheme of things?
Because he knows that Prob-Bending is popular and his success can be heard across the city. This can't happen if they don't fight him. His chance for success is denied momentarily. And what's more, it's also a perfect defense. Why? Well what can Amon do except try to make it look like he scared them? Nothing. And Amon cannot spin things that way. Korra flat out challenged Amon to a fight and since she still has her bending, it looks to the average citizen like Amon backed down. He can't say that the council are a bunch of cowards when the public has seen 1. That a taskforce exists to deal with Amon and 2. Amon chose not to fight Korra. What looks like a moment of weakness for the council is a opportune moment for someone like Tarrlock to point out, "We were not about to risk the audience and players of the game just to look brave. We were trying to protect people. But you Amon? You're the coward here. The Avatar challenged you to a fight and you declined it. Was it perhaps not as public as you would have liked? Or are you really just that scared of this young woman?" Sure, we the viewers at home know how it really went but can Amon spin it that way without being forced into a position that leaves him at the whims of Tarrlock? Amon can say that he dealt with the Avatar and chose to spare her and Korra can say "You're a liar, why would you spar me? I'm the beacon of the bending world!" The people of Republic city doesn't know how that scene played out after all and it clearly puts a kink in his plans.
Oh, and Amon would never, EVER, attack a bending hospital. He cannot show the good side of bending. He has to show the bad and focus on it. He doesn't take the bending away from good benders. He takes it away from thugs and cheaters. The ones who abuse their power. It makes him look like the real hero. Attacking a hospital wouldn't do that.