Alfred Pennyworth: A long time ago, I was in Burma, my friends and I were working for the local government. They were trying to buy the loyalty of tribal leaders by bribing them with precious stones. But their caravans were being raided in a forest north of Rangoon by a bandit. So we went looking for the stones. But in six months, we never found anyone who traded with him. One day I saw a child playing with a ruby the size of a tangerine. The bandit had been throwing them away.
Bruce Wayne: Then why steal them?
Alfred Pennyworth: Because he thought it was good sport. Because some men aren't looking for anything logical, like money. They can't be bought, bullied, reasoned or negotiated with. Some men just want to watch the world burn.
Gender: Male Location: Bringing forth the apocalypse
Saying that Joker acted the way he did because he found it enjoyable and just wanted to 'watch the world burn' is juvenile and simplistic. In reality, his motivations were far deeper and more philosophical in nature. What Alfred says should not be taken at face value- he's simply a fallible character in the movie, not an omniscient narrator. This is part of why the movie is great; every single characters is portrayed with varying degrees of complexity, but they're all mortals.
Joker's burning of the money was symbolic to his perceived rebellion against society, which he has a very negative view of (for several reasons). He sees money as a psychological shackle that, at the end of things, holds you prisoner to a desire generated by societal factors- greed. Thus, burning money is a metaphor for being unleashed, for him; existing beyond the pointless prison that is society's moronic, hypocritical morality and devolving into the world of anarchy and chaos, which he believes is the only world in which true, fundamental, natural justice exists.
... though, as ideologically interesting this action was, him burning the money was unrealistic. I mean, c'mon, are we supposed to believe that his henchmen are attracted to his natural charisma? It's kind of ridiculous.