Sounds like a regular Bag End (re: Lord of the Rings)! I've thought about building a little into a riverbank hillside, myself, though what I have in mind is really a shallow potting shed that looks like a Hobbit Hole.
I've even been looking at culvert-things (don't know what to call them), like a huge concrete pipe sold in sections. I'd just need one section--
Don't stop at one! Grow it BIGGGGG! It won't cost much more to make it two or three sections. You will pay alamost as much for the labor or equipment to set one, as two or three. The cost of the half-pipes won't be that much. I used readily available double-t prestressed concrete panels, which span 56 feet free. It makes an impressive great room, with a 36 foot ceiling!!! There are also some guys who will custom build whatever shape you desire, using inflated ballons, and sprayed concrete...
Just make sure you face it south, with an open front (glass as much as you can.. Use a compass, but find out about magnetic deviation, for the most effective use of solar heating/exclusion. The U of Minnesota has done a lot of research on energy efficient construction, and this is a good way to go! The ground temp is the same all year round, so the only interaction, with nature, is on the south side...
You have the opportunity. Do you have the will?