Here's what the Encyclopedia of Arda says concerning the origins of Hobbits:
A mortal race almost certainly related to Men, though their origins are unknown. Their most distinguishing feature was their short stature; even the tallest Hobbits rarely exceeded four feet in height.
Originally a widespread people, hobbits were found in much of the north of Middle-earth and down the Vales of Anduin. As the Third Age passed, the Hobbits moved north and west, eventually founding the land of the Shire in III 1601.
The History of ‘Hobbit’
"On a blank leaf I scrawled: 'In a hole in the ground there lived a hobbit.' I did not and do not know why."
The Letters of J.R.R. Tolkien, No 163, to W.H. Auden, dated 1955
This is Tolkien's own account of his invention of the word 'hobbit', while marking School Certificate papers: he gives no date, but from the clues he gives, this most likely happened one summer in the late 1920's. This, then, is one of the most significant doodles in the history of literature: without it, there would have been no Hobbit, and without The Hobbit no Lord of the Rings, and without The Lord of the Rings, surely no Silmarillion. If not for those ten scrawled words, the world might never have heard of J.R.R. Tolkien.
On the face of it, the origins of 'hobbit' are easy to explain: a bored academic invents an amusing little word 'from nowhere' and jots it down. As the word became well known, though, debates began about its origins. Some doubt was even cast on whether Tolkien had invented the word himself. Probably of more importance to Tolkien himself, though, was the history of 'hobbit' within his universe, and we'll address this question first.
The Invented Etymology
Why do hobbits call themselves 'hobbits'? What is the history of the word within the world of Middle-earth? These are questions that most writers wouldn't even consider, but they gave Tolkien a problem. Most of his names for characters and places came from established languages, fictional or otherwise, and so they had a 'real' history in Tolkien's imagination that could be translated into his fictional world. 'Hobbit', though, had appeared spontaneously, and so had no history of its own. Tolkien needed to invent one.
In Middle-earth's past, the hobbits had dwelt in the northern reaches of the Vales of Anduin, and the language Tolkien used to represent that region was Old English, the tongue of the Anglo-Saxons. His task, then, was to find words from Old English that might transform over millennia into the form 'hobbit'.
The word hob (meaning 'sprite' or 'little man', as in hobgoblin) seems an obvious solution. It's a mark of Tolkien's attention to detail that he didn't use it - the word is far too young (less than a thousand years old) and was unknown to the Anglo-Saxons. Hence, the Northmen of Middle-earth wouldn't have known hob either.
The solution he chose was more sophisticated: he selected the Old English words hol byldan, or some similar variant, meaning 'to build a hole', and developed the fictional compound holbytla (plural holbytlan). It is easy to see how, over several thousand years, this could evolve into 'hobbit'.
The success and ingenuity of this solution, though, hide one inconvenient detail: 'hole-builder' is, at best, a highly unconventional use of English. One can no more 'build a hole' than one can 'dig a house'. It's noticeable that Tolkien's later works tend to interpret holbytla as 'hole-dweller' rather than 'hole-builder'. In particular, he submitted 'hole-dweller' to the Oxford English Dictionary (see The Letters of J.R.R. Tolkien, No 316). We don't know what happened next - perhaps some sharp-eyed Anglo-Saxon-speaking researcher intervened - but the modern Dictionary has reverted to the more strictly accurate 'hole-builder'.
Who Invented 'Hobbit'?
Almost as soon as The Hobbit was published, questions started to be asked about the real origins of the word. Of course, Tolkien's use of it was his own invention, but was he definitely the first to use the word? Perhaps it had already been invented by someone else? Perhaps Tolkien had come across it in childhood and forgotten the event, only to have the word reappear from his subconscious years later?
These questions seem to have originated with a letter written to The Observer newspaper, published on 16 January 1938 (see The Letters of J.R.R. Tolkien, No 25). The author, known only by the punning pseudonym of Habit, claimed that a friend remembered a fairy-tale called The Hobbit dating from about 1904. Other mentions of this tale (apparently about a rather ferocious creature) have surfaced since, with dates that vary around the turn of the twentieth century.
Was there a hobbit before Tolkien's? We just don't know. So far as we can establish, no-one has yet produced a copy of this 'proto-hobbit'. Tolkien himself, while not entirely dismissive of the idea, suggested that a similar-sounding title might have been misremembered in light of his own invented word. If an earlier hobbit ever did exist, a century has passed since it was published, so the chances of finding any proof are negligible (though if you should happen across a copy, please let us know!).
The last word on this topic came from the Oxford English Dictionary, when they decided to honour Tolkien by including 'hobbit' in their hallowed pages. For the etymology, they needed to establish definitively when the word was first used. Their conclusion effectively closes the matter:
"hobbit n. one of an imaginary race of half-sized persons in stories by Tolkien; hence ~RY (5) n. [invented by J.R.R. Tolkien, Engl. writer d. 1973, and said by him to mean 'hole-builder']"
The Concise Oxford Dictionary of Current English
(If you have any old books in your attic, though, it might be worth leafing through them!)
Of Hobs and Boggarts
Throughout northern Europe, there exists a prevailing tradition of 'Little People'. They have an endless list of names: brownies, pixies, fays, leprechauns are just some of the more common. In some regions, these beings are far more than just myths or folklore: even today, they have an effect on people's everyday lives.
Take, for example, the Isle of Man in the middle of the Irish Sea: an island with a severe fairy infestation. In the southern parts of the island is the 'Fairy Bridge', a bridge that no Manxman would cross without greeting the Little People that live there. To most, of course, this is just superstition, but there are those who literally believe that they share their island with all manner of fairy creatures. Among these is a being known as a phynnodderee; shy of humans, friendly and happy-go-lucky, hairy-legged, fond of wine and beer and given to farm-work. Sound familiar?
The Manx aren't alone, of course: from Germany, where miners are helped by friendly burrowing 'kobolds', all the way to Iceland, whose Elves occupy a ghostly realm curiously similar to Tolkien's 'wraith-world', there are similar traditions.
What's more, even their names are familiar: we've already mentioned hob, but boggart, boggard, flibbertigibbet and even Hobberdy, Hobbidy and Hobberdy Dick (these last three are listed by Tolkien himself; The Letters of J.R.R. Tolkien, No 319, dated 1971).
Perhaps surprisingly, Tolkien denies that he was influenced by this in choosing the name 'hobbit', but he seems to have embraced the tradition by the time he wrote the Foreword to The Lord of the Rings. There, he says that hobbits are 'more numerous formerly than they are today', and that they 'avoid us with dismay and are becoming hard to find'. We can only realistically see this as an attempt to marry his fictional people with the 'hobbits' of folklore and tradition.
Addendum
Since this article was written, we've discovered that the name 'hobbit' goes back far further than even Tolkien suspected. We're indebted to Mark Blanton for sending along a long list of magical beings collected by a certain Michael Aislabie Denham before the year 1895. In the middle of this list, among the 'boggleboes', 'freiths' and 'wirrikows' lies the term 'hobbits'.
--------------------------------------------------------------------------------
A story that started with an idle note on a blank piece of paper has, in the end, taken us back through thousands of years of myth and language. This is one of the great joys of Tolkien - his work has an almost 'fractal' quality. The more you examine a single detail, the more it unravels into an epic mesh of connections and complexity. The last word on this matter is best left to the master himself:
"Oh what a tangled web they weave who try a new word to conceive!"
The Letters of J.R.R. Tolkien, No 319, dated 1971