Using a flash drive is a good idea but doesn't address all possible "incrimination". The example I gave of flushing your PC's web browser cache is a good example.
On IE, you can specify where the cache goes. Put it on your flash drive. Then your browser cache never existed on your hard drive