Neanderthals lived as far down as Egypt so it's totally possible and actually very probable that you have ancestry from Northern Africa somewhere along the line.

Here's a nice quote from Wikipedia

"In antiquity, the ancestors of the Somali people were an important link in the Horn of Africa connecting the region's commerce with the rest of the ancient world. Somali sailors and merchants were the main suppliers of frankincense, myrrh and spices, items which were considered valuable luxuries by the Ancient Egyptians, Phoenicians, Mycenaeans and Babylonians. "

Pretty cool if you ask me.