Sure you can. You just can’t get PAID by the government for it unless you meet their requirements. And you won’t be taking on the foster care responsibilities of a total stranger through a government arrangement.
Do you think the Amish ever get permission from the government when they adopt or foster children?