Ulster outscored the Isuzu Southern Kings five tries to three, with their last try being scored while the visitors were down to 13 men after Tertius Kruger received a red card and Alandre van Rooyen a yellow card for dangerous tackles in the 67th minute.
Handling errors close to the tryline on three occasions also cost the men from the Eastern Cape possible tries, which could have resulted in a vastly different outcome in their final tour game of the season.
The first half was thrilling as both teams played high-tempo rugby and showed patience on attack, which saw an exciting five tries being scored.
Ulster struck first in the seventh minute thanks to a powerful dash by Stuart McCloskey on the outside, and Rob Lyttle added their second try in the 14th minute after gathering a well-timed pass out wide. This earned them an encouraging 14-0 lead.
The Isuzu Southern Kings, however, fought back strongly with back-to-back tries by Yaw Penxe and Ulrich Beyers within five minutes thanks to their composure on attack and opportunism, which reduced their deficit to two points.
With four minutes left in the first half, Springbok loose forward Marcell Coetzee nudged Ulster into a 21-12 lead when he received the ball from a ruck and forced his way through the defence for their third try.
Ulster added their bonus-point try eight minutes into the second half compliments of Robert Balacoune, who used his raw pace to clear the tryline in the corner. But the Isuzu Southern Kings hit back with a well-timed pick-and-go by Michael Willemse minutes later to trail 28-19.
The visitors, however, were dealt a double blow in the 67th minute as Kruger and Van Rooyen were sent to the sin-bin, and Ulster capitalised on this to score their fifth try from a rolling maul for the bonus-point 33-19 victory.
In the other matches, the Cardiff Blues defeated the Scarlets 41-17 at Cardiff Arms Park, Connacht beat Benetton 29-14 in Galway, Edinburgh registered a 28-11 victory against Leinster at BT Murrayfield, and Ospreys held their nerve against the Dragons for a 29-20 win in Swansea.
Ulster 33 (21) – Tries: Stuart McCloskey, Rob Lyttle, Marcell Coetzee, Robert Balacoune, John Andrew. Conversions: Billy Burns (3), John Cooney.
Isuzu Southern Kings 19 (12) - Tries: Yaw Penxe, Ulrich Beyers, Michael Willemse. Conversions: Bader Pretorius (2).