Back before the Lord of the Rings movies came out, I used to enjoy arguing with Tolkien fans.
They are (or were), I discovered, among the most argumentative people on the Internet, back in the day. Far more so than the real nerds who exchanged lists of reasons why one Star Trek captain was better than another. Until you've seen a bunch of Tolkien geeks arguing, and I swear to all that's holy I'm not exaggerating this even slightly, for weeks on end about whether Balrogs had wings or whether the Rohirrim used stirrups... you don't know what vitriol is.
At one stage I tried to play peacemaker. "It's not as if anyone was going to dig up a fossilised Balrog and prove the answer either way", I said. "Why can't we each have our own image?" Which, of course, made me no friends at all, and pretty soon I learned to stay out of those threads. It was, it seemed, the most important thing in the world to be able to correctly divine the precise image that was in Tolkien's mind when he wrote the scene. To compromise in pursuit of that ultimate truth - that would be sheer apostacy.
These were the people who had gobbled up each of the interminable volumes of Trash Reconstructed From Tolkien's Litterbox & Blotter, known in the trade as the History of Middle-earth, as well as collections of the great man's letters, reminiscences by other people who knew him, and Vague Hints His Son's Housekeeper's Daughter's Boyfriend Told This Guy Who Wrote To My Local Paper.
Occasionally in these robust discussions, someone would mention Tolkien's dream of creating "a mythology for England". I think, now, that phrase is key to understanding his work.
Tolkien was a scholar of mythology. He didn't use the word lightly or randomly. He knew that it is in the nature of mythology that there is no such thing as a definitive version. The whole point of myths is to be retold.
That's why the Tolkien and his estate, for years, made no attempt to "control" his intellectual property. Everyone and his dog copied Tolkien's basic prototypes for "dwarves", "orcs", "elves" and "hobbits" from his work, and Tolkien lifted not a finger to stop them: not, as Hollywood imagines, because he was a senile old quack who didn't recognise a gold mine when he was sitting on one, but because that was what he wanted to happen.
By the time his estate fell into the hands of "competent" (read: "evil") managers, those stereotypes had made their way into thousands of imitative books, paintings, films and cartoons, games, posters, and, of course, Dungeons & Dragons.
Where there are contradictions and inconsistencies, or incompleteness, in his books - that's not just a mistake. I'm not saying it's intentional, but it is an intrinsic part of the whole. The books themselves aren't "authoritative": they are meant to be a vehicle for debate, not its subject.
This mindset is, of course, the antithesis of modern Hollywood. To Hollywood, all stories - and to the (steadily increasing) maximum extent permitted by law, all separable literary or visual elements of those stories - are by nature private property, to be exploited and monetised ruthlessly. To Hollywood, someone "giving away" his intellectual property is not a hero or a philosopher, but merely a mug to be exploited.
And that is why no film of these books made by the present-day studio system is ever going to do them justice. Movies are produced by a process based on ideals and morality that by its very nature cannot begin to understand Tolkien. In carrying Tolkien's legacy, D&D was more faithful and more effective than any movies will ever be - because D&D is based on creating new stories, not merely the messy regurgitation of old ones. To be faithful to Tolkien, you would have to donate all your scenery, sets, designs, props, models, scripts, scores and publicity to the public domain, the moment the movies were finished. And, of course, allow anyone else to sample, recut, redub, rebuild, reimage etc. as much as they liked.
Even if Peter Jackson had no problem with that (and to do the man justice, I think he might not), the studios would never allow it. It's anathema to their business model.