Having completed a full controls test management solution implementation from inception to user rollout in approximately 30 weeks, KBR reports an overall high level of satisfaction. This assessment is derived from reports with respect to a number of outcomes related to implementation processes and objectives, including:
“User satisfaction is extremely high. Users were elated with the new application because the prior solution was very inefficient and cumbersome to use. It required duplicate entry into two systems, and neither system had all the data that a user needed to see at one time. Almost all reports were manual prior to DoubleCheck, as well, and those that were not were very limited, partly because the data captured in the system was limited. We are able to report much more information from the new system.”
Patricia Pavlick
Project Manager, Financial Controls
KBR
As noted above, the 7.5-month implementation cycle reported by KBR ranks well within the Best-Case implementation scenarios benchmarked in Blue Hill Research’s July 2015 Benchmark Report Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios (Table 5).
In terms of technical deployment alone, KBR accomplished its core operations within fourteen weeks, with only one week of delay beyond its planned go-live date. This closely matches the experiences of Best-Case implementation scenarios, where technical deployment time falls between three and four months. By contrast, Worst-Case implementation scenarios benchmarked by Blue Hill Research involved technical deployment times that fell between 11 and 16 months in length, which well exceed the total implementation cycle reported by KBR. In addition, the levels of user satisfaction reported by KBR match or exceed those reported in Best-Case scenarios.
KBR’s ability to achieve an implementation within a time cycle and at levels of organizational satisfaction that match the Best-Case Scenario benchmarked by Blue Hill resulted from a number of aspects of the organization’s approach. These aspects include, among others:
Many of the practices demonstrated by KBR echo practices employed in Best-Case implementations identified in Blue Hill Research’s July 2015 Benchmark Report Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios.
Practices identified in this report include:
Blue Hill’s assessment of KBR’s strategic decisions, methodology, and outcomes reveal additional contributing factors that resulted in the success of its implementation. Table 6 summarizes these contributing factors as they related to organizational components, strategic planning considerations, operational excellence, and technical aspects of the DoubleCheck solution itself.
Based on its analysis of these factors, Blue Hill has identified six best practices in the organization’s implementation that directly contributed to the results realized, which are summarized in the following sections.
Arguably the most important step taken by KBR was the creation of detailed, specific, and written business requirements.
At every stage of the implementation process, the 75 business requirements defined by the vendor played a role in narrowing attention to the underlying operational needs driving the investment. As such, these requirements became KBR’s primary tool for solution selection as well as implementation planning. In the former, the organization’s requirements document helped to define its RFP questionnaire as well as its demo script and evaluation framework. In defining the solution itself, the requirements document influenced the shape of KBR’s configuration specifications as well as its UAT test plans. The requirements document even assisted in KBR’s efforts at user role definition, workflow design, and data property models, all factors that are often left to deployment stages and can substantially slow the implementation. While KBR did not have these elements of the solution complete prior to implementation, the early identification of requirements and processes provided an established framework that anticipated much of the effort involved in this later stage.
“If your requirements are really precise, the rest flows from there. You’ll have a roadmap that you will use all the way through the implementation. I recommend creating detailed, specific written business requirements that identify all the data elements you want to capture and the values you need for each. Everything else starts from there. You’ll have your roadmap for everything from system selection to design to acceptance testing.”
Patricia Pavlick
Project Manager, Financial Controls
KBR
At each stage, KBR was able to take advantage of prior efforts to identify both the functionality it needed as well as the non-functional architectural and delivery methods that would permit it to effectively achieve its goals. Its ability to quickly work through vendor selection and to adhere to an ambitious deployment cycle also rolled out, directly and indirectly, from the adherence to clear and unchanging requirements.
From this standpoint, it is important to recognize that the technology investment was not a transformation vehicle. Rather, KBR had already analyzed its SOX controls test and review processes and made the strategic changes it identified. The technology layer was implemented solely as an enabler to those changes. This approach is consistent to the implementation planning practices highlighted among Best-Case implementations in Blue Hill’s Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios Benchmark Report.
Companies planning similar investments will benefit from modeling their own approaches on KBR’s requirements planning process. The organization dedicated significant time to this aspect of the implementation (approximately one month), prior to any major review of software options. Special note should be taken of KBR’s “no surprises” approach, which attempted to identify factors that would impact the success of the implementation. The impact of this step on subsequent activities cannot be overemphasized.
A recurring theme throughout KBR’s implementation was the identification of the Financial Controls Group as the primary owner of the program. The Financial Controls Group possesses the authority and responsibility for SOX controls assessment and thus represents the natural owner for determining the solution requirements that would best support those operations.
Often, a functional line of business owner will be only one member of a committee responsible for an implementation. In other scenarios observed by Blue Hill, other stakeholders are brought in last minute when approvals are needed, rather than at times when they have the ability to influence planning or strategy. For example, consider situations where line of business stakeholders identify a desired solution and then attempt to “farm it out” to IT or other departments that may understand the organization’s technological ecosystem but lack deep insight into the business processes to be supported.
“We were trying to plan a new system for our users. It was important for us to have a simplified, user-friendly approach to the system interface and our workflows. We have hundreds of users and frequent changes in those user roles. The system needed to be intuitive and capable of being picked up by a new user quickly and without extensive training. Both early in the process and toward its end, we involved key business users to understand their needs. That went a long way to getting us to the right place.”
Steve Vontur
Director of SEC Reporting and Financial Controls
KBR
By contrast, KBR primarily located implementation decision making and execution with a single team: Financial Controls. The Financial Controls Group held primary responsibility for a wide array of activities, including business requirements definition, solution evaluation, project management, user acceptance management, training, rollout strategy, and other activities. Where other stakeholders were involved, it was most often to provide guidance on the decision, rather than to take responsibility for elements of decision making or execution of implementation activities. It should be noted that involvement of outside stakeholders, be it business users, KBR’s Chief Financial Officer, or IT, occurred early in the process to set constraints or guidelines, such as budget or technical requirements. This approach effectively served to empower the Financial Controls Group to act within boundaries that served the larger organizational needs, while preserving its ability to identify a solution that would best fit its business operations.
As noted above, Blue Hill Research identified this “early advisory” approach as a key practice in Best-Case implementations in the Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios Benchmark Report. However, for a variety of reasons, the exact approach employed may not fit all organizations. Organizational boundaries or procurement policies may require the use of a different process. Alternatively, aspects of the solution or process change considered may insert limitations. For example, if the organization does not opt for a vendor-hosted offering, considerably more involvement from IT will be required. Whatever the options available, however, early solicitation of guidance and alignment of requirements to organizational constraints will play a key role in effective scoping of the implementation.
In addition to obtaining guidance from key stakeholders in early stages of its investment and implementation planning, KBR also secured executive-level support from the CAO. In part, support of the CAO, along with the CFO, represented a gating requirement for the initial investment approval. However, the involvement of the CAO in this implementation went deeper. The CAO worked at early stages of the process to help align and gain support for the investment business case. Throughout the process, the CAO also helped to support the implementation by helping to identify strategic needs and prioritization as well as provide feedback on data elements and workflow components to be addressed by the DoubleCheck controls management platform.
The involvement of an executive champion is a frequent component of a successful implementation of a major enterprise technology investment. The role taken by the CAO here provides a prime example of the impact that executive stewardship provides, by securing support and buy-in from other organizational stakeholders and helping to define the importance of the investment to the business. Continuing involvement also helps to maintain focus on the implementation effort.
This is particularly important where an implementation is led by a business operation owner, as momentum for the implementation effort is often at risk of taking a backseat to the owner’s primary responsibilities. Involvement of the executive champion thus often assists in ensuring that implementation activities are accomplished “to spec” and on schedule.
Several contributors to KBR’s success resulted from characteristics of the solution and deployment model that they selected. Similarly, the ability to successfully operate to a tight implementation schedule was a result of the selection of a vendor that was able to align on project goals and to collaborate to meet them.
The DoubleCheck GRC platform selected by KBR is vendor-hosted and rests on a highly configurable application architecture. These factors alone removed two major time investments of an implementation: the application environment provisioning and the customization of the application. Blue Hill Research’s August 2015 Solution Landscape GRC Vendor Implementation Success Strategies identified these factors as key vendor contributors to implementation success (see sidebar). Similarly, the selection of a vendor that provided a flat fee for implementation services and demonstrated a willingness to embrace KBR’s requirements and implementation plan helped to ensure that the implementation could be executed to plan without scope creep or unexpected delays.
Vendor Contributions to Implementation Success
Blue Hill Research’s August 2015 Solution Landscape GRC Vendor Implementation Success Strategies identified vendor and solution capabilities that correlated to Best-Case implementations, including:
- Rapid Deployment Programs
- Application Configurability
- Availability of Out-of-the-box Functionality
- Software as a Service (SaaS) or hosted delivery models
- Subscription-based pricing
Blue Hill’s prior research has identified how these sorts of factors represent vendor supplied contributors to effective implementation. However, these characteristics are not ultimate guarantors of success in and of themselves. Solution attributes such as configurability or hosted deployment are factors to assess and trade off against other benefits and drawbacks of a solution. As such, KBR’s selection of these capabilities does not represent a best practice.
Rather, the best practice to be emulated is the organization’s attention to aligning solution architecture and delivery considerations to business and operational needs in planning the implementation. This has been mentioned above with reference to requirements definition. It bears repeating in the context of solution selection.
KBR identified its business requirements, operational preferences, and a go-live target intended to permit use of the platform to support its 2016 controls test review. It then identified non-functional attributes, such as the application architecture and hosting model, that would help further these needs. In so doing, KBR ensured that the solution and vendor that it identified possessed attributes that would facilitate its implementation objectives, rather than present compromises or challenges to work around.
In addition to including factors with an impact on implementation, KBR demonstrated a strong emphasis in solution evaluation on ensuring that vendors provided concrete demonstrations of their ability to meet KBR’s needs. To this end, one may observe the emphasis that the organization put on vendors’ ability to demonstrate how it would address requirements in RFP answers and in the demo script and evaluation methodology. KBR disfavored pre-packaged presentations and sought concrete demonstrations of how the offering could address KBR’s requirements.
“We required the vendors we evaluated to provide specific responses regarding our business requirements, not only whether or not the system met them, but how. When you get an RFP response from a vendor, most give you ‘yes’ or ‘no’ for an answer. The truth is a vendor can often answer ‘yes’ honestly when the way that it is done makes you want to say ‘no’.”
Patricia Pavlick
Project Manager, Financial Controls
KBR
This “show me” approach plays a crucial role in ensuring that an organization is able to evaluate a vendor’s true capabilities and enabled KBR to gauge not only what functionality the vendor could effectively deliver, but also how much effort would be required to implement that functionality as a working solution for KBR. A solution that looks good in a “canned” demo or “checks all the boxes” in an RFP might require extensive work and application-level customization in order to adapt it to the buyer’s needs. Uncritical acceptance of vendor answers and a lack of attention to the concrete details of how a solution delivers its functionality will result in scope creep, compromises, and workarounds that retro-fit processes to the limitations of a tool. All of these outcomes work to lengthen deployment and erode value, sometimes by dramatic margins.
KBR found that the willingness and ability of DoubleCheck to configure its platform to meet its requirements within the demonstration provided a clear signal of the vendor’s ability to effectively accommodate its needs within its implementation window. In assessing other vendors, KBR observed an unwillingness to attempt to demonstrate the specific functionality required by KBR, as set forth in KBR’s demo script, which left open questions about whether the systems could be configured to KBR’s specifications quickly and easily. In this way, how a vendor responded to KBR’s requests provided qualitative indications regarding the potential working relationship with a vendor and limitations in its system. To this end, KBR observed that the insights it obtained into the DoubleCheck architecture and vendor relationship in the demo proved to represent key attributes that helped drive the speed and success of its deployment effort.
Once again, readers should observe how KBR’s development of detailed and complete requirements that attached to both functional and non-functional attributes of the solution permitted KBR to apply this level of scrutiny.
Without a clear articulation of its requirements, KBR could not assess how a vendor was able to track to these needs. Nor would a vendor have sufficient information with which to provide meaningful responses.
No matter that the technical attributes of the DoubleCheck solution required less labor than a less configurable solution, KBR’s 3.5 month technical deployment cycle did not happen on its own. Rather, KBR identified a comprehensive project plan that accounted for all steps necessary to achieve its implementation goals and made effective use of standard project management practices to ensure that plan was followed.
Two major project management practices can be credited with the most impact of the operational effectiveness that KBR demonstrated in its technical deployment:
“Our approach was classic project management. You need a dedicated project manager, where the project is clearly that person’s first priority, on both the client and vendor side. You need to develop and use a detailed, task-level project plan and timeline. This is your project ‘bible’. You will adjust it when necessary, but it helps you to know what is needed to do and see where you need to adjust efforts to make your target.”
Patricia Pavlick
Project Manager, Financial Controls
Organizations are unlikely to require extensive reminders of the value of good project management. However, several elements of KBR’s strategy deserve particular attention:
Within a 7.5 month cycle, KBR was able to progress from the identification of a solution investment need and developing a business case for the replacement of its legacy SOX controls system to running a live replacement. By any measure that Blue Hill has tracked, KBR’s implementation stands out as a model of success. This impression is echoed in the organization’s own assessments of its experiences and resulting satisfaction with its platform and its implementation experience.
The steps that enabled KBR to accomplish its objectives represent a mix of careful and rigorous planning, vendor evaluation, vendor partnership, and disciplined focus on business process and needs as well as familiar best practices related to executive sponsorship and project management. Organizations that look to KBR’s example to extract lessons for their own implementations will benefit from each of the best practices identified in this report. However, the importance of the development of precise and comprehensive business requirements cannot be understated. The organization’s effectiveness at all subsequent stages of the implementation resulted, directly or indirectly, from the efforts that KBR put into these requirements at the outset of its project. While Blue Hill has identified effective requirements generation as a best practice previously, the forethought and level of detail seen in KBR’s example is rare. While organizations are not guaranteed to see the same results in their own implementations, learning from these practices will help them to prevent issues related to scope creep, downstream compromises, and budget and schedule overruns that cause GRC implementations to become failures before any solution goes live.
Review deployment comparisons regarding timing, user adaption and acceptance, business user impact, SOX Controls assessment administration and management impacts, and reports creation and consumption
Compare Implementation outcomes to benchmarks
Analysis of Implementation Best Practices demonstrated
Review factors contributing to implementation effectiveness
Importance of obtaining Executive-level support
Adapt a Show-Me approach to vendor claims
Review formalized Project Management practices
Review key observations and takeaways