Mutual misunderstanding there - I assumed that your original post referred to Pakistani etc Muslims as ‘Middle Easterners, since few British Muslims come from the Middle East - most are from Pakistan, Bangladesh and to a lesser extent India.
The origin of the British ‘Asian’ usage dates from the days of the British Raj, when the only people from the continent of Asia seen in any numbers in Britain were Indian. ‘Asian’ wasn’t then - and isn’t now - used for the Far East of the continent.
I see from your profile that you are British.
In American usage, the term "Asian" refers to people of a certain racial type, who are chiefly from eastern Asian countries: China, Japan, Mongolia, Philippines, etc. They also used to be called "Oriental" which somehow has become racist.
India, Pakistan, Sri Lanka, and a couple of other countries are located on the Indian subcontinent. Most Americans, if asked to name countries of Asia, would be unlikely to mention any countries that are on the Indian subcontinent because we don't typically think of them as being part of Asia.
The Middle East stretches across North Africa and Eurasia, based on the fact that the countries in this region are typically Muslim.
As I think about it, I think American geography is based more on similarity of ethnicity than on physical landmasses. It seems, from my viewpoint as an outside observer, that British geography is based more on the physical location than on a primary ethnic characteristic of people living in the area.