In 1963 Prime Minister BJ Vorster introduced the General Laws Amendment Bill which would permit the detention in solitary confinement without trial for 90 days. One particular clause was directed as a single individual: Robert Sobukwe. He had been due for release, but instead was transported to Robben Island, where he stayed in 24-hour solitary confinement in the yellow house on the left for six years.
The other buildings are kennels which housed the Prison's guard dogs.