Snagging List

cyberyacht replied on 05/01/2017 08:59

Posted on 05/01/2017 08:59

PLEASE READ BEFORE POSTING.

This thread is not for discussions. Post ONLY factual information.

To provide a comprehensive list of problems encountered, I would suggest that each post lists only one issue at a time.

Provide details of problem  as concisely as possible.

Operating system, machine used and relevant software. i.e. Macbook/ PC/tablet, Windows XP/ iOS7 etc, Safari/Internet Explorer/Chrome inc version No.

Please do not enter into discussions such as "I've had/not had this problem" as this will detract from the objectivity of the thread. Hopefully this thread may assist the IT department in finding where "all the bodies are buried".

PLEASE READ BEFORE POSTING.

cyberyacht replied on 30/04/2020 16:47

Posted on 30/04/2020 16:47

From Justins post perhaps the problem lies in the multiplicity of suppliers involved. Left hand not knowing what the right hand is doing. Too many cooks etc.

huskydog replied on 30/04/2020 17:00

Posted on 30/04/2020 14:58 by Justin Bricknell

Hi all 

I work for one of the CAMC IT teams; I'm not directly involved in maintaining the Club's website but I'm involved in the technical calls regarding the current issues. 

What I can say is that we work with multiple partners (hosting companies, external web development partners, API partners) as well as internal coders and infrastructure teams. Whenever the Club's website has serious performance or functionality issues, a Major Incident is declared internally and various IT teams get together for the initial investigations and triage stage. Depending on how those investigations develop, will determine which external partners need to be engaged but these incidents are always taken seriously. 

The issues in the past couple of days followed a new code release for various bug fixes from one of our external partners. Shortly after the release, we were made aware of issues with some of the replies to forum threads and Vouchers in the Club Shop section. Investigations concluded that, with regards to the discussion threads, the replies were still there in the back-end database but weren't being indexed/updated correctly and therefore weren't displaying on the front-end website. It was decided a re-index should fix this issue and was planned for last night, as this needed a website outage. A temporary fix was put in place for the Vouchers while the website was live yesterday afternoon. 

The manual re-index of the Community (Club Together) was completed at approx 2am and the website was available again at 2.36am this morning. This followed testing (and confirmation) that some of the missing replies from the day before were now appearing, as expected. 

This morning, we checked back on the Club Together pages to see if members were still experiencing any issues. It became clear that although the missing replies had returned, there were still some issues regarding user profiles, missing threads and some news stories. We were then later made aware of some functionality issues on the UK sites search pages. None of the above were included in the post-indexing testing last night, as they were not issues that were reported the previous day. 

Several technical calls have since taken place today and we are trying out various fixes in one of our non-production environments at present. If any of those fixes are successful, there may be another website outage this evening. We will put up a maintenance page on the website if this is required. 

I will post a further update when I can (it may be via Rowena). 

Once again, apologies for the issues currently being experienced but we are working with our external partners to resolve these as soon as possible. 

Justin 

 

Posted on 30/04/2020 17:00

That’s a long winded way of saying “we haven’t got a clue” ........

Oneputt replied on 30/04/2020 17:09

Posted on 30/04/2020 17:09

I just don’t understand what is different about the club site when compared with say big banks or international companies.  I have never known my bank to be taken offline for maintenance. The last company I worked for had 40k employees over all the continents, due to my job requirement I had 24 hour access even outside the office and again I never encountered any shutdowns.  

So come on what is really different about the club site 

obbernockle replied on 30/04/2020 17:10

Posted on 30/04/2020 15:47 by JVB66

Thankyou for your input, and is now much more transparent  with the information you have posted ,about what and who  is involved

Posted on 30/04/2020 17:10

Thanks from me too. The snags are resolved so that is good. A technical statement is an excellent first response, but only a first response. Presumably there was (is) an inherent root cause which is responsible for the many issues? Why the clubs web site is so bad may be known to the it team. I suspect the it team hasn't actually acknowledged how poor it is, but in any case the head of the it team is responsible. Productivity of the it team must be dreadful. The club's management board should know what is wrong, and what to do about it.

 

 

SteveL replied on 30/04/2020 17:54

Posted on 30/04/2020 17:09 by Oneputt

I just don’t understand what is different about the club site when compared with say big banks or international companies.  I have never known my bank to be taken offline for maintenance. The last company I worked for had 40k employees over all the continents, due to my job requirement I had 24 hour access even outside the office and again I never encountered any shutdowns.  

So come on what is really different about the club site 

Posted on 30/04/2020 17:54

I get alerts from my bank every month or so, warning that the mobile banking will offline for maintenance / update,  usually on a Sunday night.

Justin Bricknell replied on 30/04/2020 18:11

Posted on 30/04/2020 18:11

Thanks for all your responses to my earlier post. 

As I mentioned earlier, I'm not personally responsible for the maintenance of the Club's website on a day to day basis, and generally only get involved in these issues when there is a Major Incident. 

I can't give definitive answers to all of the comments made but I'll try to reply to some of the comments. 

@Cyberyacht - I don't think it's uncommon for an organisation of CAMC's size to use multiple partners to maintain it's IT infrastructure. We, like many others, only have a limited amount of internal resources and require external support in certain areas.

@Oneputt - I can assure you that most organisations with a complex website will need to put their site into maintenance mode at times. Indeed, my bank (Metro Bank) have a 6 hour outage planned this weekend.

@Obbernockle - generally speaking, in IT incident management, the priority is to restore the affected service. Sometimes, this may be as simple as restarting a server. Once the issue is fixed, we'd look at conducting Root Cause Analysis to establish why it happened and put in steps to make sure it doesn't happen again. The current issues with the CAMC website are complex and there are many individual elements that need to be resolved. The investigating teams have been told to restore the missing sites in the UK Club Sites section of the website as a priority. Once these are restored (hopefully within the next few hours) we will then look at the remaining issues and conduct an RCA.

I know my posts won't satisfy everyone on these forums but I'm just trying to share what I can with you, be as transparent as possible and reiterate how seriously we are taking the current issues. 

Some of the teams involved were working on yesterday morning's release (7am start). In my last post, I mentioned that the initial issues were fixed at 2am the following day - with some of those same people involved. Some of those staff are still working on the issues at the moment and will be involved in a 6am release tomorrow, which is a pre-requisite code deployment ahead of the UK Sites Rolling Inventory release next Wednesday. We really are working flat out to get the issues resolved as soon as possible. 

Justin

 

brue replied on 30/04/2020 18:22

Posted on 30/04/2020 18:22

Thanks Justin, you're all doing your best, much appreciated, what would we do without all this chat on CT?! wink

PS My bank has frequent maintenance breaks too, people working through the night to fix and update.

By the way if you come across any excess self raising flour in the works please send some my way...laughing

Tinwheeler replied on 30/04/2020 18:43

Posted on 30/04/2020 18:43

Cheers, Justin. It’s good to be kept in the picture.

Yes, my bank too closes its website for maintenance. Usually on a Sunday and always with notice.

nelliethehooker replied on 30/04/2020 19:48

Posted on 30/04/2020 19:48

There's still a problem with booking a club site. I've just gone onto the booking page and selected "Region" and the only one that comes up is Cotswolds. When I try to select a county is come up with "County" but no list of them. If you select Sites in England all one gets is the Cotswolds, and selecting that no list of sites in that region appear. When I try to select a county is come up with "County" but no list of them. However if one uses the Search and Book facility all the regions and counties are listed.

This topic has been locked, no new replies can be added.

near Malvern Hills Club Campsite Member photo by Andrew Cole

Book a late escape

There's still availability at many popular UK Club campsites - find your perfect pitch today for a last minute trip!

Book now
Woman sitting in camping chair by Wastwater in the Lake District with her two dogs and picnic blanket

Follow us on Facebook

Follow the Caravan and Motorhome Club via our official Facebook page for latest news, holiday ideas, events, activities and special offers.

Photo of Wast Water, Lake District by Sue Peace
Visit Facebook