Sahih Muslim, Book 15, Hadith 502 (Muslim 3267)

Al-'Ala' b. al-Hadrami reported Allah's Messenger (ﷺ) as saying:

The Muhijir should stay at Mecca after performing the rituals (of Hajj) but for three (days) only.