Last Minute Cancellation

We booked a 5 night stay at a CL. All packed up an ready to go, when the weather dramatically changed to gale force winds very heavy rain and some foggy patches. After much deliberation we decided to cancel our trip as the forecast was the same for the duration. My question is should we pay for the time booked, or agree to visit in the very near future? The site manager said we did not need to pay them and that all their bookings had been cancelled. We really feel bad about this as it is their living. What should we do?