Pollard delivered the killer blow five minutes from full time with his fourth penalty goal of the match, finally edging Wales, who twice came from behind to level the scores.
The Springboks held a 9-6 lead at the break and when Wales levelled matters early in the second half, the match and a place against England in the final was up in the air.
The impact of bench players proved decisive in the final quarter though as they won three crucial penalties that tilted the game towards South Africa.
The opening quarter saw a battle for territorial dominance with the boot and that aerial battle delivered no clear winner. One of Faf De Klerk’s early probes forced a knock-on though and from the resulting scrum and South Africa attacking, Wales went offside.
Pollard slotted a first penalty to give his side the lead.
The Welsh started to carry the ball and from such an attack, the South African defence went offside as well, giving Welsh flyhalf Dan Biggar a chance to even matters out.
A knock-on by Wales from the restart gave the Springboks an opportunity to flex their muscles in the scrum and earned a penalty. Pollard slotted it with easy to regain the lead after 20 minutes of play.
Another Welsh knock-on from a high kick handed the Springboks their next attack, with Damian de Allende running strongly. From a resulting line-out the Springbok maul earned good meters, but the attack broke down.
This enabled some territorial advantage though, with the Welsh under pressure in their own half. Another maul, after 33 minutes of play, handed the Springboks their next opportunity as it was dragged down by Wales. Pollard made it three out of three with an angled kick and pushed the lead to 9-3.
The Welsh lost Thomas Francis (prop) with a shoulder injury from the restart after a heavy collision with Duane Vermeulen, but there was something to smile about just before the break for coach Warren Gatland as Biggar slotted a second penalty.
Both teams still had everything to play for coming out of the halftime huddle. Wales got the board rolling early in the second half with Biggar slotting a third penalty goal after another lineout infringement by the men in green and gold.
The Springboks made some handling errors under the high ball early in the second half and that stilted their momentum on attack. Rassie Erasmus called on his bench within the hour, with the front row and Eben Etzebeth all replaced.
The new front-row forced a scrum penalty and from the resulting attacking line-out the Boks finally got the first try.
Malcolm Marx made the first surge of the back of a ruck, before Pollard broke the line and De Allende drove over in the corner in the 56th minute. Pollard kicked the conversion from a wide angle.
The Welsh forced an attacking lineout but after at least 20 carries, but could not find a way over the line. They kept attacking though, electing to scrum when the Boks were penalised again.
That bravery was rewarded as the Boks were outflanked on the blind and Josh Adams went over in the corner after 65 minutes. Leigh Halfpenny kicked the conversion to level matters again.
The Welsh kept up the pressure but a crucial steal from Francois Louw gave the Boks the ball back and when Wales was penalised from the resulting lineout, Pollard stepped up to put his team up again with five minutes to play.
The last frantic minutes were played out in Welsh territory and when South Africa won another scrum penalty with no time left on the clock, Pollard could gratefully kick the ball into touch.
Springboks 19 (9) - Try: Damian De Allende. Conversion: Handre Pollard. Penalty goals: Pollard (4).
Wales 16 (6) - Try: Josh Adams. Conversion: Leigh Halfpenny. Penalty goals: Dan Biggar (3).