Dune was actually more complex, with more "palace intrigue" than LOTR. Plus, the theatrical Dune movie was cut too much to be any good (and having David Lynch direct it was a big mistake). Every indication that has come out on LOTR indicates that the makers have come very close to realizing on film the excitement and atmosphere of the books. Some characters have been cut (Tom Bombadil, Glorfindel). But overall, the movie appears to be what it needs to be: visually spectacular and exciting. If the movie is only 80% as good as the books (considering that virtually no movie is ever as good as the book), then it will be incredible.
One element in favor of LOTR already: most of the actors are very, very excited about how it's turning out. Plus, the LOTR movie trilogy is going to run 8-9 hours when it's all done. There's a lot of room for complexity in that amount of movie. Dune can't be told in two hours (the Sci-Fi TV version, while lacking in special effects, was much better than the theatrical version).
It may not make a lot of money (it won't be a "Blair Witch Project"), but it won't be a flop.