According to reports from the Boland club, Joseph was playing for his club's second team when he complained about chest pains. He was replaced and treated by medics next to the field, but he passed on shortly afterwards.
Mr Alexander said: “Our sincerest condolences to the family, friends and team-mates of Ansley Joseph.
“It might be of little comfort to his bereaved ones, but on behalf of the South African rugby community, I want to wish them strength in this difficult time. They are in our thoughts and prayers.
“Most of our clubs in South Africa have an intimate relationship with their players and communities, as was the case here, so we hope all those involved can find some solace and peace knowing that Ansley was playing the game he loved so much at the unfortunate turn of events.”