Quasi-independent evaluation: a hybrid model inspired by ExxonMobil’s OIMS

Whenever I disclose that I started my career as an engineer at ExxonMobil, I get odd looks. ExxonMobil’s environmental reputation is so contentious that there is a whole Wikipedia page on the ExxonMobil climate change controversy.

Besides its position on climate change, the company’s environmental reputation still suffers from the Exxon Valdez oil spill (1989): the spill has changed the oil industry forever. What most do not know is that the spill also changed ExxonMobil.

After Exxon Valdez, the company’s senior management scratched their heads in trying to find answers. How could such an accident happen? And most importantly, how could similar accidents be avoided? Gathering leaders from all its operations around the world, the company developed a new Operations Integrity Management System (OIMS).

There is always a bright side to every shadow. I dare to say that OIMS is ExxonMobil’s environmental treasure. Unfortunately, the company does not disclose many details behind its decision not to pursue recognised international standards such as ISO 14001 and adopt its own OIMS instead.

Fortunately, having worked with one of the Senior Managers that developed OIMS, I was able to learn first hand why the company chose a customised evaluation system, rather than a one-size-fits-all system with externally certified auditors. The main pillar of OIMS is that the ownership of its 11 elements (see picture) must lie with the staff involved in operations, rather than the supporting Safety, Health and Environment (SHE) department. SHE is the guardian of the system, but all staff are responsible for it. As for the system’s evaluation framework (the 11th element), there are two key principles:

  • no-one knows better what works and what doesn’t than staff in the field; and
  • independent evaluators are often treated with suspicion by staff and thus are unable to read between the lines.

Combined, these mean that it is easier to hide information from independent evaluators than from staff from the same company, who know the ins and outs of the processes. Lastly, independent evaluations are expensive, making internal evaluations a no brainer.

The challenges with internal evaluations include the intrinsic staff bias and the possibility of collaboration between staff to hide problems from senior management. In OIMS, these are addressed by forming evaluation teams with staff from different units and framing the evaluation as a learning opportunity. Considering all results are kept “in-house”, teams feel more comfortable in sharing problems.

The lessons from OIMS stayed with me. After all, as Lavoisier stated over two centuries ago, in this world “nothing is lost, nothing is created, everything is transformed”. Now, fifteen years after I left ExxonMobil, I incorporated OIMS principles into international development evaluation.

Inspired by OIMS, questioning the widely accepted value of independent external evaluations using the OECD’s DAC criteria, I created a hybrid approach: a quasi-independent evaluation where the evaluator team is formed of both, staff and an external consultant. Jointly the team creates a customised framework to assess the programme.

Recently, I carried-out a peer-evaluation, going beyond participatory approaches. I trained and led 11 donor and implementation agency staff in a Whole Person Evaluation and jointly we evaluated an entrepreneurship and life skills training programme for young women in Sierra Leone. The results were fantastic.

The process provided not only a learning opportunity for all those involved, but also a safe space to openly discuss aspects of the programme that could be improved. Relationships were strengthened, critical findings were immediately acted upon, resolved and documented. The team could rapidly move on to use the results to inform and shape the programme strategy going forward. If you want to know more, drop me a line. It will be great to roll-out this quasi-independent evaluation hybrid model to other programmes.

Isabela Souza

Your Name (required)

Your Email (required)

Your Message