Sunan Abi Dawud, Book 22, Hadith 10 (Abu Dawud 3252)

Sa'id ibn Ubaydah said:

Ibn Umar heard a man swearing: No, I swear by the Ka'bah. Ibn Umar said to him: I heard the Messenger of Allah (ﷺ) say: He who swears by anyone but Allah is polytheist.