Arriving at the implemented solution from the recognition of investment need is the result of a journey that begins with scoping need and business case and ends with technical implementation and rollout. Invariably, these processes are complex. Often, they are long, extending to a year or more of effort. Notably, KBR accomplished all of these stages over an approximately 30 week program (Figure 2).
To place these experiences in context, Blue Hill Research’s July 2015 Benchmark Report Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios identified the median time for technical deployment alone as 10.5 months.
By contrast, the organization’s experiences are comparable to Blue Hill’s benchmarked Best-Case Scenario, which measured three month technical deployment cycles at the shortest edge of its range. KBR’s ability to successfully complete the full implementation without reducing scope within such short cycles is an outcome of good planning and project management, selection of a solution that could accommodate its needs, and effective decision making throughout this process.
“Our main objectives came down to ensuring we had an integrated solution. We were overburdening our user community with multiple ancillary applications that we couldn’t bring together in the legacy solution. There were two key components to this objective: because of our adoption of a peer-review process, we had to have a well integrated workflow from the various tester to peer reviewer roles and other administrative type functions. We also needed to be able to store all related test results and supporting documentation within the new solution, bypassing the need to extract information from multiple systems of record.”
Steve Vontur
Director of SEC Reporting and Financial Controls
KBR
The following sections will trace the considerations and choices made by KBR at each of the major stages of the implementation process, including:
- Business Case & Requirements Definition
- Vendor Evaluation & Selection, and
- Deployment and Rollout
Figure 3 summarizes the key stakeholders and their corresponding roles and activities at each of these major stages.
Review of the approach taken by the organization and the outcomes that resulted in its implementation process will permit organizations planning similar investments to extract best practices and structure their strategies to help generate similar levels of success.
Business Case And Requirements Definition
KBR dedicated approximately one month to developing a list of approximately 75 business and technical requirements for the new platform. It then identified approximately fifteen of these items as “key requirements”, resulting from a systematic review of SOX test and review processes and related reporting needs.
These requirements are illustrated in Table 2 below, based on their alignment to the major technological challenges identified by the organization.
The Financial Controls Group, as stakeholder responsible for the accuracy of SOX assessment, served as the primary decision maker in this process. However, it gave weight to the needs of business users and executive management that would use the tool. The process was characterized by the following priorities:
- The business process and information needs of the peer review process was given high priority
- Information consumption patterns and reporting demands of business leadership was given high priority
- Technical requirements, such as compatibility with KBR’s IT infrastructure, received low priority once KBR determined it would be using a vendor-hosted offering
- Flexibility to accommodate change in organization, process, and control wordings received high priority
KBR adopted what it describes as a “no surprises” approach to its business requirements definition and evaluation of vendor responses to its RFP. This approach resulted in a set of highly specific business requirements, which would later inform the solution specifications, and a high degree of scrutiny given to vendor responses and demos.
In parallel with requirements generation, the Financial Controls Group created a business case justification for its investment, using a standard form used for organizational IT purchases. Development of the business case itself, which echoes the challenges identified above, resulted from consultations between the Financial Controls Group, the CAO, the CFO, and select business users of the platform.
Approval of the investment required disclosures to and consultation with IT as well as the sign-off of the CAO and CFO. KBR reports that IT involvement at this stage was minimal due to the selection of a vendor-hosted option. The investment approval required the satisfactory review of the written business case. This involved analysis of the following factors:
- Intended business benefits related to SOX controls risk
- Operational scope of the implementation and the intended use case
- Success criteria
- Expected cost of the investment based on preliminary quotes sourced from sample vendors
Once the business case was approved, KBR identified a set budget for the investment based on the range of expected costs presented. Further financial approval was not required as long as the cost remained within the margins of the approved budget.
Solution Discovery And Selection
The Financial Controls Group led KBR’s initial vendor discovery. This process involved a series of informal online searches and review of analyst reports, consultations with business associates using similar systems, and initial phone conversations with known vendors. After the discovery period, a select set of vendors demonstrating roughly comparable functionality was identified for a request for proposal (RFP) process.
“We invested a significant amount of time up-front on requirements, knowing they would be the key to selecting the right solution. The relationships we built with vendors in the early phases of the search were less important than what the system could deliver. If the vendor didn’t show how a specific requirement would be met in a demo, we went back and asked them to try again. If they couldn’t, it was an ‘X’ on the scorecard. It was clear to us from the final scorecard which vendors would be able to deliver.”
Steve Vontur
Director of SEC Reporting and Financial Controls
KBR
RFP Process
The Financial Controls Group worked with the organization’s Procurement Office to initiate and execute the RFP process. KBR gave vendors approximately six weeks to submit responses to questions addressing:
- Functionality
- Application cost and total cost of ownership
- Financial viability of the vendor
- Support offerings and processes
- Technical platform and maturity
- Customer retention and reviews
- Length of vendor experience in SOX support
KBR determined the assessed non-functional factors based on the risks it foresaw in implementing a new solution and entering a new vendor relationship. Again, the Financial Controls Group played the primary role in determining these factors and in reviewing the RFP submissions of vendors. In addition, IT identified key questions to include with respect to technical characteristics to be assessed. In drafting its RFP questions and reviewing the responses of the vendors, KBR placed primary focus on how vendors identified their ability to meet its 75 requirements. As such, KBR ensured that its vendor review would focus on how effectively the vendor would be able to deliver a solution that satisfied its needs, rather than simply stating their solution possessed the capabilities desired.
Vendor Demonstrations
Approximately two weeks after vendors submitted their responses, KBR scheduled demonstrations of the solutions with four vendors based on their evaluation of the RFPs. The demonstration process was intended to permit KBR to see how the vendor would structure and configure its application to meet the organization’s requirements. For this reason, KBR developed a detailed “demo script” design to show KBR specifically how each system would address each of the organization’s business requirements. KBR instructed each vendor to follow the script as closely as possible. To create a formalized and consistent evaluation methodology, KBR developed an evaluation template and ranking system that its demo participants would use.
KBR’s evaluation factors and priorities at this stage closely mirrored those of the business case and RFP evaluation processes, albeit at a more granular level of detail. Primary factors for review included:
- Functionality
- Ease of use
- Implementation process
- First-year cost
- Total cost of ownership
KBR favored vendors that could demonstrate, in person, how its requirements could be implemented and configured in the product. For this reason, the organization disfavored slide-based presentations or a prefabricated “standard” demo presentation. Again, KBR gave significant consideration to how changes to controls, business processes and organizational structure would be handled by the system and the projected resulting impact on a closed assessment cycle.
After dedicating two weeks to demonstrations, KBR spent a day compiling evaluation results. The organization then identified the top two performers in the demonstration phase for additional discussions, which occurred over several weeks. As with the RFP evaluation, the Financial Controls Group was the primary reviewer in the vendor demonstration stage. IT stakeholders participated as well to assess technical issues and identify potential concerns, such as platform instability, system design obstacles, and other contributors to negative performance.
Solution Selection
On the balance of its evaluation factors at each stage, the organization selected DoubleCheck to provide its SOX management platform. KBR reports that this decision reflected its assessment that the vendor demonstrated its ability to address its 75 business requirements, but stood out by demonstrating within the confines of the demonstration how to configure the solution to meet those needs, where other vendors indicated that weeks or months would be required to customize their offerings to meet the organization’s requirements and use case. The selection was made by the Financial Controls Group, as the cost of the solution was within the approved budget and IT involvement would be minimal to deploy a vendor-hosted platform.
Deployment And Rollout
Translating KBR’s business requirements and the raw capabilities of the DoubleCheck platform into a working solution supporting SOX controls testing constituted the major component of the deployment process. As KBR would be adapting a new tool for existing operations, items such as controls definitions, tester and reviewer assignments, and major process requirements had already been established.
Responsibility for the remaining effort would be split between KBR and DoubleCheck and executed in parallel. KBR was responsible for defining its user roles, workflows, data models, and reporting schema. The Financial Controls Group bore primary responsibility for developing proposed models for these system attributes, with review and feedback from the CAO and guidance on implementation from DoubleCheck.
In addition, DoubleCheck bore direct responsibility for provisioning the underlying application environment and configuring the application to KBR’s specifications. The Financial Controls Group retained primary responsibility for overall project management, user acceptance testing (UAT), and rollout and training.
Table 3 summarizes these respective roles and responsibilities for each of these set of activities within a RACI matrix.
Table 4 summarizes the approaches taken by KBR with respect to key components of the implementation process. KBR’s objective was to use the platform to support 2016 SOX controls testing. Accordingly, it set out to complete these steps within a four month window. When completed, the organization had run over its target by one week. The definition of clear, detailed requirements at the program’s outset provided the groundwork for the items in Table 4, as well as a basis from which the vendor and KBR could execute to these goals.
“We had a collaborative relationship with DoubleCheck. They gave us diagrams of proposed system workflows for controls tests and issue management, which we modified to align with our business processes. The starting point was the old process, but we were open to changing if it made sense, and we did make some changes. DoubleCheck performed the configuration of the system according to our specifications. They were easy to work with and didn’t have the budget creep issues of some vendors. They were focused on making sure we were happy with what they sold us. That’s a big success factor.”
Patricia Pavlick
Project Manager, Financial Controls
KBR
Additional contributors to success resulted from both the vendor and KBR’s approach to deployment, such as:
- Application Contributors: The DoubleCheck application is highly modular and supports a deep level of in-application configuration of most aspects of the implementation with little to no customization or changes to code. The use of a vendor-hosted delivery model contributed, as well, by eliminating the need for new infrastructure and environment creation by KBR.
- Vendor Relationship Contributors: DoubleCheck also executed its aspects of the implementation under a flat service fee, rather than by hourly or labor-based pricing. This worked to prevent budget creep in the implementation as well as incentivized efficiency on the part of the vendor. This model also has the advantage of minimizing mid-project negotiations regarding the respective parties’ responsibilities and the scope of billable work.
- Project Planning Contributors: The Financial Controls Group managed the execution of all parties’ activities and project schedule. At the outset of the implementation, the Financial Controls Group created a detailed project plan, which fixed responsibility for each task and set a target due date. The Financial Controls Group identified its necessary milestones and implementation scope based on the functional capabilities it needed to support its core supported business operations on the go-live date.
- Project Scope Contributors: Capabilities that KBR required in the platform but did not need to use on the go-live date, such as reporting features, were reserved for updates that would follow the primary implementation. After setting the scope and major milestones, KBR worked with DoubleCheck to get input about the nature, timing and sequence of tasks that were specific to an implementation of their system. It then plotted tasks accordingly to fit its milestones. Where possible, activities were scheduled to run simultaneously.
- Project Management Contributors: KBR closely managed all aspects of the project plan under formal project management practices. KBR designated a formal project manager from the Financial Controls Group with a peer on the DoubleCheck configuration team. KBR held standing project meetings twice a week. In addition, it held periodic project plan review meetings where the project management leaders from both KBR and DoubleCheck could meet. These meetings were used to identify gaps in execution, identify potential issues or obstacles, and reassess milestones or refocus priorities. Accordingly, the organization reports that it was able to remain aware of crucial needs and respond accordingly to prevent slippage or unforeseen delays.