Polls that ask that specific question are being run by people for whom the issue is one of primary importance, by pollsters and sites who promote that issue.
The last general poll that I saw which listed the immigration issue as an option to check off, had it come in below marijuana legalization.
Many of the issues that Americans are concerned about are directly or indirectly affected by illegal immigration. For instance, most Americans would say that they're more concerned with finding a job than they are about stopping illegal immigration. But many jobs are filled by illegal immigrants, which makes it more difficult for many Americans to find a job.
Most Americans would say that they're more concerned with their children getting a quality education in their local public school system than they are about stopping illegal immigration. But one of the reasons that many American children aren't receiving a quality education is because many schools are filled with illegal immigrant children who aren't well versed in English, so it takes more time for them to learn, and this slows down the learning process for the American students because the teacher has to go at a slower pace. Also, having all of these illegal immigrant students in our public schools adds to overcrowding in the classrooms, which also affects the quality of education that American students are receiving.
These are just two of many issues in which Americans would probably say that they're more concerned about a particular issue than they are about stopping illegal immigration. But many of these other issues, just as with jobs and quality of education in our public schools, are also affected by illegal immigration.