Travellers moved onto the site of Aegal House in Aylsham at teatime yesterday (Tuesday), a year ago to the day since they last came to the town.
Police said they were notified of travellers on land on Burgh Road at 5.40pm. Officers monitored the site and the group moved on at 2am this morning (Wednesday).
Norfolk Police are working towards serving a civil notice on travellers who have set up camp at Aylsham Lodge Hotel.
A spokesman said police were aware of a traveller encampment and were working with partners, the landowner and their representatives.
Officers will continue to monitor the situation, but are satisfied that no criminal activity has taken place at this stage.
About five vehicles, including caravans, were reported in the car park of the hotel, which closed late last year.
A spokesman from Broadland District Council said they were making efforts to contact the owners of the site because it is private property.