The origin of this phrase comes from a time where the Moors, inhabitants of north Africa, used to invade from the sea and sack coastal villages in Spain. The phrase means that there are no Moors at the coast, which generally meant that there are no problems coming our way.