Sunan Abi Dawud, Book 2, Hadith 557 (Abu Dawud 947)

Mu’aiqib reported the Prophet (ﷺ) as saying ; Do not remove pebbles while you are praying; if you do it out of sheer necessity, do it only once to smooth the pebbles.