I'm not a geologist but I've spent 45 years studying rocks and minerals and know a little about them. Here's one way a square hole could be formed in a rock (and it's basically the same one AppauledAtAppeasementConservat suggested).
A number of minerals crystallize in the cubic (isometric) system and some of them form nearly perfect cubes. One of these is iron pyrite (iron sulphide) -- a likely candidate for this situation because it can be easily dislodged from its host rock matrix or chemically degraded. I've found square holes in rocks where pyrite crystals were formerly located. But there are other cubic minerals that could do the same thing.
If it's a hole left by a crystal the host rock is igneous or metamorphic material brought to Mars' surface by the meteorite impact that made the crater. It's certainly an interesting object and I hope the rover takes a hard look at it and science comes up with an answer. The fuzzy-looking stuff on the top of the larger rock to the left needs some examination too. I've seen rock formtions like that and they generally indicate some sort of hydrothermal action like geysers -- if I'm right it could be a real indication of Martian water, past or present.