Aishah narrated that:
The Messenger of Allah said: "Whoever stays with a people, then he is not to fast without their permission."