There is a precedent. When Ricks College was expanded into a four-year school and renamed Brigham Young University-Idaho, the intercollegiate athletics program was scrapped. (I am told that BYU-Idaho has one of the best intramural programs around.)
If the elimination of athletics is seen as too drastic a step, BYU-Provo might follow the lead of the University of Chicago, one of the founding members of the Big Ten:
[President Robert Maynard] Hutchins heaped scorn upon schools which received more press coverage for their sports teams than for their educational programs, and a run of disastrous seasons gave him the trustee support he needed to drop football in 1939. The decision was hailed by many, but few other schools followed Chicago's lead. (Office of the President, University of Chicago)In 1946, the University of Chicago withdrew entirely from the Big Ten. The university resumed playing football in 1969, and now competes as a member of the University Athletic Association (UAA) in Division III of the NCAA.
Playing in Division III has had no discernible effect on the reputation of the University of Chicago. Some 85 Chicago faculty and alumni have won Nobel Prizes. Personally, I would prefer a Nobel Prize to a national championship any day.