Queens Park started the brighter and should have gone in front on seven minutes after Mark Ferry blazed over from seven yards.
Elgin's first chance of the match fell to Martin Johnstone after Stuart Cumming's cross was headed down by Jamie McKenzie but the striker could only put over.
In the 33rd minute Elgin took the lead against the run of play. Adam Nelson took a free-kick on the left and Steven Reilly was deemed to have pulled Johnstone. The referee awarded a penalty and Mark Booth made no mistake.
Elgin nearly doubled their lead on 37 minutes when a Jamie McKenzie free-kick was met by the head of Hugh Dickson but it went inches wide.
In the 47th minute Elgin had the first chance of the second half when McKenzie played through Booth and he was blatantly took out of the game by Michael Dunlop.
He had already been booked and should have been shown a red card but no action was taken. Nelson took the resulting free-kick but watched his effort fly just past the post.
In the 66th minute Queens Park deservedly drew level. Ferry cut in from the left and produced a terrific save from Kieron Renton but unfortunately for the Elgin keeper Dunlop reacted quickest to slot into the empty net.
Ten minutes later Elgin were nearly back in front after Cumming's 30-yard drive smashed the bar.
In the 75th minute Queens Park nearly took the led after a scramble in the box was pounced on by David Weatherston but his shot rebounded back off the post.
By this point Queens Park were well on top and Weatherston should have done better when he was one-on-one with Renton but his shot flew over the bar.
Queens Park's winner came on 84 minutes when Alan Trouten delivered a corner and Anthony Quinn was quickest to react to head home from six yards although there was some doubt if had crossed the line.
Queens Park deservedly held on in the dying stages to take all three points.