Managing risks in a production environment, that is making money for customers, is extremely essential. However, most often, due to the unpredictable nature of the production support or operations management work, the fear of unknown increases drastically.

More often than not, for an operations analyst or a production support analyst every day is a new day and every problem is a new problem and hence the traditional risk management model that suggest to Identify->Analyze->Plan->Track->Control. The traditional model assumes there is a significant time available that will allow you to analyze and assess the risks after you identify it. However, in the production support or operations management area, the time is something that is not available and you are expected to react it quickly.

I have been part of a workshop recently to discuss about the Risk Management and how it could be done in such a volatile, unpredictable and unknown environment such as production (or Live).  In one of my previous experiences about awarding the winners in an organization, it was observed that the companies, most often, tend to reward the people who do better crisis management than the people who do better risk management and that often means that the risks are tend to be reacted only when they are realized and become a bigger problem.

So, at the end of the discussion, it was more or less agreed that the Risk Management in a production environment is all about behavioral change and mindset. Interesting ? .. read ahead !

If you consider the possible responses to a risk once you identify it, they could broadly classified as follows,

  • Terminate – terminate the risk at the source and do not accept the same
  • Transfer – transfer the risk to the concerned stakeholders and ensure they are mitigated
  • Treat – accept the risk immediately and start controlling
  • Tolerate – accept the risk and do nothing !

If you revisit all the scenarios you had experienced related production support or operations business, they are more often than not demand urgent attention.  A priority 1 ticket is waiting or some incident is threatening to take the shape of a bigger problem.  Now, for such situations, can you terminate the risk ? Can you tolerate the risk or can you transfer the risk and keep quiet ? I would think no ! In all such cases, you would have taken quick action to either resolve the risk yourself or ensure that the risk is resolved at the earliest.

Now, coming back to my earlier statement of relating the Treat, you would agree that to treat the risk in an production environment that requires collaboration across multiple teams, you need to develop the ownership & risk taking mindset. Someone needs to take the ownership and drive the problem through to the solution or mitigate the risk in full.

Few tips on mitigating the production risks are as follows,

  • Keep customers informed of more bad news than the good news. Even if you do not believe, the customers are more prepared to listen to worse news than you can possibly give the.
  • Expose your vulnerability without going into victim mindset !

Do you manage the teams that work 24×7 across the shifts? Especially on 12 hour/day shift and 4 days/week model? If yes, then I would really be interested to know how manage to have team meetings and to the team collectively?

Getting everyone together for a briefing, team meeting, gathering, round table etc., has been a great concern because of the lack of people in office (due to their shift working). Almost half of the team is off on weekly offs every day while half of remaining work in day and rest in the night shift. Thus at any point of time, I have access to only 1/4th of the team at wok (again due to shift working).

Since most of the team members are not available having a team meeting is really out of question and does not really add value. So I would need to develop some method of getting the offline updates across to everyone and make them equally participant in the decision process, which, may last longer than usual one / two hour meeting. It could actually take 1 week depending on the shift schedule.

Few things I have been contemplating of implementing are,

  • A discussion forum – Creating an online discussion forum where each discussion topic is listed and kept open for a period of 1 week.  Offline reminders are sent to the team to go through the discussion forum and the comments / questions are invited and are answered through further replies / comments over the forum.
  • Offline recording of meetingsRecord the meeting discussions in audio / video formats and again share with the team offline. Invite comments over the recording & engage them into question / answer session offline. Create a Question Basket where you invite questions and get them answered through email replies / audio recordings or video sessions.
  • Issue tracker & project update acknowledger – Create an online application where the important decisions are stored and compel the team members to acknowledge that they have read and understood these decisions. If they have issues, they can raise queries back to the manager via online application.

There might be few more things I could possibly do, but I would really like to know if you have any good suggestions which I could take on board and try and implement them so as to work even more effectively.

Please leave a comment, a feedback or a note on my blog if you could help.

Ohh .. wait a minute !

This article has nothing to do with any award / reward function you would see on a television, cinema, but its more to do with the way awards and rewards are judged within an organization and how sometimes it puzzles me.

Today, I had been to a rewards committee meeting as a guest judge and at the end of meeting, I left the room with lots of thoughts to ponder with.

award-cup Few days before the meeting, I was sent an excel sheet with good detailed nomination information about  6 candidates from whom I needed to judge the top two candidates. The top two would be rewarded a certain amount as per the company policy. There is a quota of the monthly / quarterly awards which are distributed at the end of every month / quarter.

Normally, the rewards committee has few fixed judges and few guest judges. I was one of the guest judge this time and was asked to voice my inputs on who would, in my opinion, win the award and why.  During the discussion, I observed that the rewards committee makes a good effort and attempt to discuss each case and try to give their opinions on the face value of the nominations they are presented.

On a whole, the process of inviting nominations from Project Managers and having a meeting in which a representation from Sr Management, HR & few guest judges would decide who wins, is good enough in most of the cases.

Most of the decisions, unless the judges know the nominee well, are made on the basis of what is written in the nomination form and how well the arguments are written.

The whole discussion made me wonder, in the end, who actually gets rewarded?

Is it the person who has done a great work or the person who has actually written a great nomination form with appropriate arguments?

How do you ensure that the right person gets the award?

What I have also seen, is the fact that the award normally goes to the person who has come out of adversity and saved a project from a serious escalation or from a difficult situation. Example of this could be someone who has just handled a business critical Priority 1 problem and resolved it within the SLA timings.

Then there is one person who makes an effort all year to keep a consistent performance and ensures that escalations do not happen and also ensures that the crisis situation never arises. This person, although gets nominated for the award, but would never get it, because his nomination form would be a lightweight, compared to the first one.

Typically in the support project, what I have seen is the team or the individual that resolves most problems or most crisis situations is rewarded the most. While, in a positive way it is correct to appraise a person who has gone extra mile to resolve the crisis and save the project from any potential penalty.

However, on the other hand, few people who are proactive, keep wondering what they need to do to get a recognition? In few discussions with the teams previously, we used to joke that we have got the project to such a stable phase, that there is no more challenge left which would give the team a chance to shine above the rest. !!

In a nutshell, there are quite a few factors that would determine whether a project or a person would get a reward or not and few of them I could list as follows,

  • Type of project – If a project is stable and all processes are matured, then there is a less likely chance that the project would be considered for any award. Eventually, all the people working in the project have less chance of getting recognized for their work.  Effectively, since the project itself does not offer any challenges, even outstanding person would struggle to get a recognition for the work there.
  • Nature of work – If someone has saved the project some costs, efforts or time, then the person would be worthy candidate of being nominated for the rewards. Whenever the nomination is given, its important that all the necessary details are furnished properly. Moreover, the benefits must clearly come out of the nomination form. The words such as costs reduction, resource reduction, automation, efforts saving, crisis resolution  should come out appropriately and should be emphasized.
  • Glorify the work -  As I mentioned above, irrespective of how many judges know your work quality personally, the decision is made on the basis of what is written in the nomination form. So its very important to glorify the work and presented accordingly.
  • Awareness – One of the most important aspect of being rewarded for outstanding work, is being aware of rewards process and ensuring that the information is proactively sent with some good details as I mentioned above.  During few discussions I had with the teams, it was observed that the team ‘expect’ the managers who work with you to ‘recognize’ their work automatically without the team expressing willingness and desire to actually have one awarded. Its a misconception that the work would automatically be awarded without any advertisement or internal marketing. Unfortunately, its one of the most difficult part for teams to understand and practice.
  • Due attention by project managers – Its very easy for project managers to put the blame on the rewards committee about lack of reward to the team. The rewards committee is the soft target by many project managers who fill up the nomination form just because its mandatory and lack ‘awareness’ as I mentioned above.  The project managers should ideally go all out to ensure that all correct details are furnished, glorified and presented accordingly for the rewards meeting.

These are just some of the tips you could use to assist yourself in presenting your case better.  Finally, a quick word on what I believe,

There is only one alternative to hardwork, its name is ‘smart work’ !

Its often expected by customers that the vendors take ownership and full accountability of the projects and related pitfalls. While on one hand its good to take ownership of the tasks in your project and deliver them on time, within budget and exceed expectations, its equally important to understand the remits and scope of your work and project before assuming the responsibility & the accountability of failures.

Not only by the customers, but the display of the ownership from team is demanded by project managers & senior managers as well.

A Dilbert strip below just about sums up how the management assumes project teams to “own” things !


Quite rite ? Do you think so ?

It is essential, to demonstrate the capability to own and deliver the things, however it is equally important to know the limits of the ownership and a clear understanding of what you should deliver and be accountable for. 

If you are into application support and maintenance projects, its even more important to know your project boundaries & scope so you could actually define the components you own and maintain.

Typically in the support projects, with the scenario of multiple support teams supporting various components in an end to end journey, the problem or fault can lie in any one or more components in the chain.

If any service impacting incident occurs in such scenario, I have seen customers pushing the front end (or Portal or Website) support teams for solutions and RCAs. Irrespective of where the problem or issue lies. It is fact that in a web based journey the problems are only visible in browser and thus generally it is perceived that the teams that supports the actual portal application are responsible for owning the investigation & resolution of issues.

Generally it is OK for the ASGs to take up the responsibility of owning the issues & try to get a solution in place ASAP. However, it is probably too much to expect them to own the end to end stack and push for investigation & resolution where the fix lies outside the boundary of the support team.  In such cases, I would rather have the E2E Service Managers to own the fault and drive the resolutions through the downstream systems rather than the poor portal ASG teams. 

I do not think there is any concrete definition or rule which says which tasks / issues / problems / escalations you should own up and which you should not. However, in case of the conflict I would suggest it is always better to stick to the rulebook (i.e., scope of work, application boundary) and take a wise decision.

After all, you would not want to lose brownie points !!

Today, during one of my team review meetings, one of my colleagues raised an idea of having a chat server hosted on intranet to allow the team members collaborate with each other for issuing and requesting updates on various incidents, problems and their progresses. A good idea to have one chat server, but few team members were probably not pro to the idea due to the ‘push’ nature of the chat and they did not want to be disturbed with many messages broadcasted to the team if they were not of any interest to them.

Then the obvious solution to this idea was to install ‘twitter’ like microblogging software on one of my intranet server and allow teams to follow rest of the team to ‘pull’ the updates as and when they need it.

To my surprise, when I explained the concept of microblogging to the team and gave an example of ‘twitter’ to the team, none of my team members seem to know ‘twitter’. This was a shock to me, especially when my team was supposed to be pro in Information Technology !!

Nevertheless, explained them the concept of microblogging to the team and asked them to explore ‘twitter’ concepts and find out how it works.

laconica Upon doing the search myself, I found best suited to our needs of having an online microblogging and collaboration tool for the team.

I would still need to get the team explore this product and get a pilot installation tested on our intranet Linux server but by the looks of the product, it looks to be exactly what I needed for my team to enable better collaboration.

Anyways, few main expectations I am having from this kind of collaboration tool being used in the support project is to enable the disparately located team put up updates on a simple question @What are you doing?@.  Obviously my team is located at various locations across India & UK so having a centralized tool would certainly help us in better collaboration.

Another important usage I envisage of this tool is to give updates to customers about any major release updates or change implementation. It is obvious that while the teams are busy doing the installation of a software on the production servers, they would not like customers or any other teams asking the questions on status updates and disturb them.  So if we have this kind of tool, it would definitely enable us ‘post’ proactive updates on intranet which could be followed by various customers / teams to find out the updates themselves.

Thus, a job for me tomorrow is to find out someone in my team who could take on the job of doing the R&D and install the software on my intranet server and take it from there ..