Muwatta Malik, Book 45, Hadith 9 (Muwatta Malik 1731)
Malik related to me that he had heard that when Umar ibn Abd al- Aziz left Madina, he turned towards it and wept. Then he said, "O Muzahim! Do you fear that we might be among those that Madina casts off?"