But today, where majestic palaces once stood, crumbling bricks remain. Faced with a growing population and increased motorists, many of the town's historic buildings have either fallen down or are in a deteriorating condition, leaving huge gaps in Stone Town's narrow winding alleys.
In the five decades since the 1964 revolution, Zanzibar has become unrecognizable says fashion designer and repatriate Farouque Abdela. After living in the UK for years, Abdela describes seeing his hometown upon his return as nothing short of traumatizing.
"There was nothing to remind me of my childhood," he says. "Everything that I remember was more or less gone... I felt someone had played a trick on me.
"There were new buildings, the streets had changed. It was like being a tourist in your own home."
Abdela is using his standing within the community to help educate Stone Town's inhabitants and restore rooms and houses in the old town. Among his projects has been the redesign of the Emerson Hurumzi, a boutique hotel that once belonged to a powerful merchant and adviser to the sultan.
"Some people live in historical buildings but they don't know it," he says. "To them it's just a building. Maybe if they are made aware..."
The restoration men
Othman Rashid Omar is another artisan making people aware of Stone Town's heritage. Using traditional materials and techniques he has trained over 70 people in the art of restoration.
"It's very important to train young people to restore the old buildings," Omar says. "I am 52 years old and once I am gone I am gone, so we need more young people to know how to protect the old buildings and protect the future."
Part of his remit is the restoration of the beautiful ornate wooden doors ubiquitous throughout Stone Town. These doors are intricately carved, and according to the local rule of thumb, the more detail the wealthier the owner.
The main challenge, is cost. It's expensive to restore an old building, less so to build a new one, with many of the original architecture requiring hard to source materials. Convincing locals to invest is a crucial part of his efforts.
"To preserve and promote"
Abdela is conserving local culture through other outlets also. Western clothes are creeping into the lives of Zanzibar's residents, pushing out the traditional Kanga, a colorful type of printed dress.
Commonplace on the archipelago, Kanga also served a separate purpose in years past, becoming a canvas for spreading political ideals and values from the 1920s onwards. But today most Kanga is produced in India and China, Abdela says. Kikoi, a male garment worn around the waist like a cummerbund, is now made entirely overseas.
The fashion designer is encouraging young artists to continue the old traditions of dressmaking and supporting artisans like Mussa, 39, one of the last still holding the flame.
"It could have died out without him carrying on," Abdela admits. Now Mussa employs 11 people, training a new generation of fabric makers.
"To preserve and to promote" is Abdela's mantra.
"Africa is not about bare chest[s] and grass skirts, you know. There is more to it," he says. "There is opulence, there is glamor there is beauty... I think the future is bright."