Really a toss-up between the first two and a separate toss-up between the last two.
I always say Casino Royale is the best Bond movie, but Skyfall is the best movie with James Bond in it.
I prefer Quantum because it was a Bond movie that deconstructed typical Bond tropes: no sex with the main Bond girl, CIA was a tertiary villain of sorts, Forster went with some long takes (James crashing a speed boat into a larger boat to rescue Olga's character), the shootout at the opera house with no audio but the music, Craig never utters, "Bond. James Bond". However, it still paid homage to The Spy Who Loved Me with the tie grab henchman drop at the opera house. The worst part was the opening super quick-edited car chase and the 20-yard parachute opening for survival. Overall, it's kinda-disjointed, but I really enjoyed when it hits its highs, one of my upper half entries in the franchise, if not a top-tenner.
Spectre, while probably the most traditional Bond film of Craig's, is average-to-good throughout, with a great opening sequence and the best fight in the series with Mr. Hinx on the train. It's last act, though, is not that good at all, and somehow both Moriarty from BBC's Sherlock and Christoph Waltz as the big bad Blofeld himself were completely underwhelming and underused.
Casino Royale is the definite #1. That's a legitimately good movie that actually made me interested in James Bond.
The others are kind of a toss up. I remember enjoying Quantum of Solace more than many people and didn't really like Spectre much at all except for that opening sequence which was really awesome. Skyfall was nicely filmed, but I found it sort of overrated. That being said I'm actually not a huge Bond film fan and it's been a long time since I've seen any of them.