They don't see it as erosion of their power, they see it as being able to impose what would be unpopular law without the blow-back of accountability.
I agree with you that they are uncomfortable with the concept of accountability and find it easier to allow excesses than to fight against them. However...there is undoubtedly a steady and increasing erosion of congressional power caused not only by agency dictates, but also by the increasing reach of executive orders, presidential memos, and frequent “adjustments” to laws (e.g., constant changes to Obamacare). Laws were not intended to be open-ended and infinitely modifiable to suit the political convenience of one party.
There is so little regard for congress’ authority that agency bureaucrats routinely have no qualms about lying to it. So what’s the point of having a congress if its not going to do its job? Why bother voting? They clearly don’t want the responsibility.