^"Weak, hurt and exiled" is irrelevant because the next line he clearly made a disclaimer and said "but were I in the fullness of my might, I would not" because such course of action is "neither right nor WISE."
Thor judged the situation and backed away from it, not purely because he thought it wasn't right, but also because he didn't think it WISE.
This is the immediate proceeding page. Note how Thor is trying to convince him not to do this, and questions whether BRB can succeed in the endeavor. (please log in to view the image)
It's the next page where Thor declines to go with BRB, even with if he were at height of his powers. It's not just because Thor "doesn't think it's right." It's because he's CONCERNED about the whole situation. (please log in to view the image)
__________________
"It is unwise to battle the Power Cosmic." - Silver Surfer
But he still doesn't say anything close to him being afraid. He just said the action is against the course of nature. He doesn't believe it's the right thing to do. Thor has gone up against Galactus several times in the past, don't see why a more powerful version of Thor would all of a sudden be afraid of him. Still, I'll drop it since I don't want to spam up the respect thread.
And apparently when stuck in Thano's machine he had to draw upon his reserves of power to survive and they are gone and he has been trying to eat regularly since to rebuild his core and the slight break Bill has caused weakened him catastrophically to the state he is at in the picture above,Surfer goes on to say at any normal time like before Annihilation it Bill's plan would have taken months.
In this pic Surfer says when Galactus dies all life within 42 lightyears would be scourged.
After Galactus' shields were shattered, Juggernaut's son throws Spider Girl, American Dream, and Wasp(after they've shrunk down) towards Galactus' ship/tech: