Well, it's a bit of a fuzzy concept in ordinary use, but in math, it refers to the indepedence of parameters of an equation.
Let's make up a chesslike game with a more indicative example. Suppose the rook can only move a number of squares related to, say, how many squares he last moved a bishop.
In the original chess rules, the number of squares the rook can move is orthogonal to the behavior of the bishop, in our new game, it is not. The idea of the word extends beyond merely physically perpindicular, and is, by my lights, a pretty useful and compact word.