It's not a loan.
But you start out your example by stating:
Say you loaned out $1,000,000 at an adjustable rate, say 3 month Libor plus 2 points.
Is this a loan?
You proceed to state:
You go to another bank that is willing to pay you 5% fixed for the next 3 years in exchange for your adjustable payment.
Is this a loan? If not, how is this swap structured? Can you walk into a bank and tell them "I'll trade you my income stream lasting 3 years based on 3 month LIBOR plus two percent for a flat 5 percent per month, and the loser pays the difference"?
No.
If not, how is this swap structured?
If one party made a fixed rate loan and the other made an adjustable rate loan, they can swap the payment streams. Or, they can swap a payment stream without actually having loans outstanding.
Can you walk into a bank and tell them
No, you can't. If you worked on a trading desk for JP Morgan you could call Goldman Sachs and set up a swap. If you worked for a hedge fund, you could call an investment bank and set up a swap.
You can build a derivative based on almost anything. A derivative derives its value from something else. An option traded on the CBOE derives its value from the performance of the underlying stock or index.
An interest rate swap derives its value from the performance of some interest rate index. Could be based on the 1 year US T-Bill or Libor or Fed Funds. Some derivatives are pretty commoditized. Some might be one-of-a-kind.
Is that any clearer?