Ok, my answer is not going to be that reliable because, I must admit that I have only seen the flm once (despite it being fantastic) but I clearly remember hearing him say that line in the film.
I can imagine the way he says it and the look on his face...just not where he is at the time. However, like has been previously said, I think it is when he has to walk the plank.