Again, implied and inferred. That he meant, directly and unequivocably, 'less intelligent' has been inferred, but he has DIRECTLY said that he never meant genetically inferior, which seems more clear to me.
And even if he did mean it- I very much doubt there is any basis for you saying it is a. baseless b. racist and c. only a hypothesis. About the worst thing that could be said about it... would be that he is wrong. But you cannot go crying foul every time an expert makes a comment upon his field, the consequences of which you do not like.
And he certainly should not be silenced about it.