It never particularly bothered me about Leiter being played by a Black actor. In Connery’s comeback, “Never Say Never Again”, Bernie Casey played Leiter, and did an excellent job as an equal to Bond (previous Leiters were generally played by desultory actors, with only Jack Lord being a stand-out in “Dr. No” and David Hedison in his two turns). I thought Casey was the best overall.
As for the notion of Idris Elba playing Bond, why not come up with an original film and character specially-tailored for him now that they can parlay into a 3 or 4 film series ? Is Hollyweird so completely bankrupt of new ideas that they’re trying to shoehorn a Black actor into the role of a White Scot secret agent ? It’s so ridiculous they might as well get Gabourey Sidibe to take over the Jason Bourne character.
To use a starker example, the script for The Honeymooners remake was mediocre and the film sucked, but I had no problem casting a black actor as Ralph Kramer. The audience can buy that there's a working class guy from Brooklyn named Ralph Kramden who drives a bus for a living, whether the actor playing the role is white or black. Furthermore, I'd argue Cedric the Entertainer did a good job and was faithful to the role as Jackie Gleson envisioned it. The problem was the script, not the actor.
On the other hand, James Bond's backstory in the source material makes it extremely unlikely the character would be black. Fleming's bio for James Bond is that he is:
"the son of a Scottish father, Andrew Bond, of Glencoe, and a Swiss mother, Monique Delacroix, of the Canton de Vaud. The young James Bond spends much of his early life abroad,becoming multilingual in German and French because of his father's work as a Vickers armaments company representative. Bond is orphaned at the age of 11 when his parents are killed in a mountain climbing accident in the Aiguilles Rouges near Chamonix."
What I've noticed about casting in the UK is their roles are even less accurate to the demographics of their country that US casting is. You see a lot of black actors from England cast in major roles. If you had to guess the racial makeup of the country by looking at their fictional roles in movies and film, you'd probably guess the UK is around 10% black. In reality, the overall UK population is only 3% black (compared to the overall US population being 15% black). Scotland is also far less black than the UK as a whole: according to the last census, there are 6,247 black scots, or 0.12% of Scotland's population. A non-white MI:5 agent would be much more likely to be a British person of Indian ancestry, but you see far more black British characters than east Indian British actors. The odds of an MI:5 agent with a scottish father and swiss mother being black is minuscule.
In addition to the unlikelihood of James Bond being black, both Idris Elba and Adrian Lester are a bit too old to be starting a career as Bond. There's a reason why Daniel Craig was cast at 37 and Sean Connery was 30. Fleming envisioned Bond as being in his "mid to late 30s" at the time the stories are taking place. Idris Elba is 42, Adrian Lester is 47, and Craig is currently making another Bond film so the earliest they'd take over the role would be around 3 years from now. Roger Moore was the oldest Bond ever when he started the role as 46, compared to Idris Elba being around 45 and Adrian Lester around 50 when they started -- that's too old to play Bond for 3+ movies. Moore was way out of place playing Bond at 57.
Many films is the past decade have ridiculous stunt casting that gives a middle figure to the source material: Jamie Foxx playing the Daddy Warbucks character and Benedict Cumberbatch playing "Khan Noonien Singh" were totally unbelievable, for example. But I have a sad feeling they could cast Cheech Marin as Pippi Longstocking, and a bunch of fanboys on the internet would defend it and say we're racist if we oppose it.