Martin Harty was both sinner and saint for Elgin as his late goal decided a dire match. He pounced on a goalkeeping mistake to net an 80th minute winner after throwing East Fife a lifeline by giving away a penalty.
Just nine Elgin fans bothered to turn up to watch their team take the lead after six minutes as the Fifers went through the motions.
Allan Dempsie's charge down the left caused problems, and Stevie Donnachie's cut-back was hammered into the net by Willie Martin.
A fight broke out in the tunnel when East Fife beat City two weeks ago, and it was David Robertson's men who showed the desire for revenge. They should have doubled up in 21 minutes, but Donnachie wasn't able to pick out the unmarked Martin in the box.
Martin did manage to force a decent stop from John McCafferty with a hookshot that had to be tipped over.
East Fife levelled from the spot after 38 minutes when Harty sent Stevie Nicholas tumbling, and the striker mis-hit his penalty past Kieron Renton.
Centre half Hugh Dickson pushed into the visitors' box and met Jamie McKenzie's cross with a header that hit the side netting after 52 minutes.
He then used his height to power another header from Adam Nelson's corner, but Craig Lumsden made the block. The Fifers briefly threatened when Brian Fairbairn sent in a dipping free-kick which just went over the bar.
Elgin pinched it with just 10 minutes left when McCafferty fumbled McKenzie's free-kick and Harty reacted quickest to net the rebound.