Overview

---

Major incidents are incidents that are used to track and resolve major losses to service. Major incident records use the same form as regular incidents with additional fields used to record information for the major incident process.

In this article: 

• Report an IT Services Major Incident
• Identify an IT Services Major Incident
• Assign Roles and Responsibilities
• Your Next Steps
• Important Information

Report an IT Services Major Incident

---

• Contact the Service Desk
  • (858) 246-4357  or  x6HELP
  • Service Desk Support Channel on Teams: https://bit.ly/3fm2ghl
  • Do not email unless contacting the service desk is not reasonable or possible
• Supply the following information
  • Description of the issue
  • List the applications or systems impacted
  • List services impacted
  • Identify the customers impacted - if known
  • Identify when the service began to experience difficulties or failed
• If a team is addressing the incident advise the Service Desk
• If no team is addressing the incident request the Service Desk engage team(s) to resolve the incident
• If there is a need to create the major incident yourself please follow these instructions: Major Incident Ticket creation

 

Identify an IT Services Major Incident

---

In order to provide customers with as much information as possible and prepare IT Services to manage high impact incidents, here are some guidelines:

• Is it a business critical service?  See list: link - only report emergencies for defined business critical services
• Is the service available?
  • No: Contact the Service Desk immediately and report the emergency
  • Yes: Contact the Service Desk and report the abnormal behavior
• If you know a business critical service is not available, there is customer impact or customer impact is imminent, contact the service desk immediately and let them know what the situation is.  The Service Desk will open a Major Incident (emergency) ticket and/or post relevant information to the status page.
• If your service is exhibiting unusual behavior which requires investigation and after 30 min of investigation no resolution has been found contact the service desk to inform them of the situation – the Service Desk will post a warning on the status page and notify internal groups there is an issue and request others to report any other service abnormalities.

 

Assign Roles and Responsibilities

---

When a Major Incident (emergency) arises it must be dealt with quickly and thoughtfully in order to minimize its impact. Here is where the roles, responsibilities and procedures for managing an emergency incident are detailed.

Roles:

• Customer portal - method for customers to report a Major Incident (emergency) to the Service Desk
• ITS Reporter - when escalated from within ITS the ITS Reporter notifies the Service Desk
• Service Desk - central point of contact and distributer of information
• Incident Manager - manages / oversees the incident and is the central point of contact between the Service Desk and Resolvers
• Resolvers - investigate and resolve the Major Incident (emergency) - communicate status and needs with the Incident Manager
• Senior Management Team (SMT) - escalated to higher levels of management as necessary and allocate resources

 

Your Next Steps:

---

1. Identify a Major Incident (emergency/outage)
2. Establish communications with the Service Desk
3. Establish communications between the Service Desk and an Incident Manager
4. Deploy individuals or teams to resolve the Major Incident (emergency)
5. Deliver notifications / updates to any and all who need to be aware there is an issue at appropriate intervals
6. Escalate awareness to higher levels of management as appropriate
7. When the Major Incident (emergency) is resolved, close out the Major Incident (emergency) ticket and include a closing message to all who need to be aware the issue has been resolved.

 

Important Information

---

Zoom conference call information

From Systems with Zoom installed: https://ucsd.zoom.us/my/itsemergency

• This is the same link added to every Major Incident (emergency) ticket upon creation of a Major Incident in ServiceNow.

If you need to call in rather than join using the above link use the dial-in information below:

• Dial-in US Toll-free: 833 548 0282
• Meeting ID: 809 773 5251

Backup communications channel: (only use if Zoom is not available)

On Teams, join the "ITS-Emergency" channel under the "Information Technology Services" Team. This should only be used if Zoom is unavailable. DO NOT use this channel unless directed to do so by management or the incident manager for the event in progress.

Direct link to Teams "ITS-Emergency" channel: https://bit.ly/3f8OsXw

Posting information on the status page

To post information on the Status Page go to 

Creating a ticket yourself

To create an emergency ticket immediately using email, select itsemergency@ucsd.edu and add relevant information as indicated above. Doing this will send notifications to over 100 ITS staff members who might respond based on what is detailed in the ticket. This is NOT the recommended method - if at all possible call the service desk at the number listed above. However, if time is critical and actions need to be taken without delay, don't hesitate to create the ticket yourself. Also - if after 30 min you haven't heard from the service desk create the emergency ticket yourself.

To report an outage to the service desk the preferred method is to call, as indicated above. If this is not possible, send email via servicedesk@ucsd.edu or use the customer portal to report "Something Broken". This will be subject to some processing time and should only be used for non-urgent situations.