The people that I know, use their phones to adjust the volume or set the “mode” (crowd, quiet, home, etc.).
How far away is your phone from you?
Mine is about 3feet away.
I'm touching it because I'm weird. Normally, it's in my pocket, but at night, it's in another room, and set to vibrate. Except for the 1 of 6 weeks when I'm on call at work. Then it's in the same room.