Wednesday, April 04, 2007

Burst

Tuesday, 3:18 PM. I am chatting with one of the Swede's in his office and my phone rings. It shows "anonymous" which usually means a call from Europe, most likely one of our customers. I answer, but the person hangs up. Knowing that the call handling system that we use calls each number and lets it ring three times and goes to voice mail after the last number is dialed, I walk back to my desk and use the handy dandy web interface to check the voice mail. As suspected, a call can from Europe, but it was from our incident mgmt group. Since they are always on line, I fire up the internal Jabber chat and ask them what is up. They tell me about a major issue that has been brewing and they want escalated. I look at the issue and call the contact to get more info. After hearing his spin on the issue, I call my peer in Malaysia to see what he wants to do about it since it will affect him more than anyone. I wake him up (it was either 4 or 5 in the morning there) and give him the information. He listens and I ask him a total of three times what he wants me to do. All three times he states that he will handle it. I acknowledge him verbally and follow up with an e-mail to him, the inc mgmt group in Europe, and the contact I spoke with earlier. Once all this was done, I finished up locally and drove home.

6:00 PM roles around and I am home, have fed the dogs, let them out, and eaten. I have collected the mail, filled the dogs water bowl and washed a few dishes. A storm was brewing so I took both dogs out to their thing before it hits. Neither want to and are more interested in the other pet owners who are out doing the same thing. After short and unproductive walk, I go back home. I was watching discovery when my mobile rings. It is from Malaysia...it is my peer in KL. He begins telling me that the issue is "global" in nature and asks me who is covering the global role since out mgr is out on holiday. I have no idea...he did not tell me anything before he left. My KL contact, Chong, says he will call the previous global esc mgr and see what she says. I got a very bad feeling about this, but told him to call me back if he needed me. The night goes on and Chong does not call me back. I wondered if this was good or bad.

The storm began to hit pretty hard. There was a little hail and the rain drops were huge. The sky looked yellow and there were tornado warnings all over. Then, as soon as it appeared, it disappeared. The yellow was gone and the dark gray of nightfall was descending.

I switched the TV over to NBC to watch Law and Order. I was beginning to have some sinus issues but I did not want to take anything. I was concerned that Wednesday would be a bad day with repercussions regarding the handling of the incident. I know that my job is more political than critical and the way it is handled is key. I am not big into politics. I am pretty "no nonsense" most of the time. This issue, while it was being actively worked on, was potentially a time bomb.

Off and on during Law and Order, the local weather man kept breaking in and giving an update which was a little overkill as far as I could tell. Since I was not worried at this point, I decided to go to bed. My wife came home from her women's ministry event shortly after that but I was too zonked out to greet her properly. I slept soundly until 2 or 3 AM, but tossed and turned after that, still worried about Wednesday.

The alarm finally went off and I grabbed the dogs and took them outside. I let the Golden use the bathroom and I let her in. I then took Little Man for his morning walk. The storms had brought in very cool and very windy weather...enough that I needed sweats instead of shorts. After a quick 20 minute walk, I came back and fixed my coffee and completed my normal routine.

I got to work early (I left 10 minutes early) since I had a training session at 8 AM and I wanted to get in and do my reports as well as check to see what collateral damage was sustained because of the issue the night before. When I got to work, I was very surprised to see that I only had one e-mail regarding the issue (now I am really worried!) and it was an FYI issue. It appears that the team working on the seemingly impossible issue (a 10 year old server that is not under warranty or supported with no spare server to replace it, running a critical app) had about 75% of the issue working via a workaround. The other 25% was going to take a bit of finagling, but it appeared that it was under control. With that seemingly good news in hand, I logged on to my training class (which would have been very good had the trainer spoken clearer English, not to mention that someone did not have their line muted and there was a lot of background noise).

After the class I went to let our incident mgmt know about the issues in China. While he had his own issues to worry about, he thanked me for keeping him in the loop.

The stress level of this situation has decreased some. I am still waiting for the other shoe to drop, but maybe things will be calm.

No comments: