You are right, but I am right also.
You see, the problem is how we define the word "magic".
In letter 131, Tolkien admits this problem, and connects it with the fall of man and fall of the angels (i.e. Melkor, the angel who did not follow God but decided to make the earth as his own fiefdom, so earth and all things in it are tainted with evil).
"I have not used 'magic' consistently, and indeed the Elven queen Galadriel is obliged to remonstrate with the Hobbits on their confused use of the word both for the devices and operations of the Enemy, and for those of the Elves. I have not, because there is not a word for the latter (since all human stories have suffered the same confusion). But the Elves are there (in my tales) to demonstrate the difference. Their 'magic' is Art, delivered from many of its human limitations...and its object is Art, not Power, subcreation, not domination and the tyrannous reforming of Creation...The Enemy, in successive forms, is always 'naturally' concerned with sheer Domination, and so (is) the Lord of Magic and machines; but the problem (is): that this frightful evil can and does arise from an apparently good root: the desire to benefit the world and others...according to the benefactor's own plans..."
The men in Tolkien who use this "power" actually have elven blood, and so it is licit for them to use it (Aragorn and the Kings are descended from Elrond Halfelven's brother, who decided to remain mortal...
the elven rings were meant to preserve and to create new things, and to obtain wisdom, not to conquer.
The bad news is that Tolkien regarded modern machinery and technology as similar temptations to evil...since they, like "magic" were being used to manipulate and control creation..
And so they are. Technology is the art of tool use, and tools are subject to the will of the user. Tools make things easier.
There are "Albert Schweitzer"s and "Josef Mengele"s in the world. The difference lies in how they used their knowledge and technology.