The bridge between IT Service support and IT audit becomes most apparent when one imagines the risk associated with failures among any of the IT Service process dependencies.  What would happen if any of the service areas were taken away? 

Suppose, for example, an IT help technician is blind to the true nature of a problem and is allowed to change enterprise configuration values based solely in disgruntled comments?  What if the problem is accurately reported, but no one takes accountability for its permanent solution?  Suppose the problem is identified and resolved, but the solution is never communicated to the helpdesk, the end user, or the business?  What if the solution creates more problems?  What if the only person who knows how to solve the problem quits or is reassigned within the same company?  What if the change is executed without notice to the business?  Worse yet, what if there is no process for change approval, and as a result a perfectly good solution is never executed at all?
Even if an incident ticket is accurately reported, and the solution is clearly defined, will someone consider that based in these findings, there is indication of necessary infrastructure change? 

Moving to a more optimistic view, consider the power of effective problem management.  Noted by countless studies of well run corporations, optimized service management leverage established IT framework (e.g., ITIL®) where managed solutions are driven by the business, events are tracked, and recovery of critical service is achieved with the support of knowledge banks and balanced risk assessment.  Classified problem details and patterns in incident response becomes the basis of a decision support system, and knowledge library.  This repository is often the basis for future planning and performance optimization projects.  Such projects would be introduced through formal process or Change Management.  Relationships and dependencies within Service Management are clarified by a review of controls associated to “Change Management”.

At the heart of Change Management, the process inputs and outputs involve enterprise control over infrastructure configuration.  As demonstrated in theses control processes, however, inputs and outputs to change are evidenced within and throughout Service Management.

Domain: Acquire and Implement    Process: Manage Changes
All changes, including emergency maintenance and patches, relating to infrastructure and applications within the production environment must be formally managed in a controlled manner. Changes (including procedures, processes, system and service parameters) must be logged, assessed and authorized prior to implementation and reviewed against planned outcomes following implementation. This assures mitigation of the risks of negatively impacting the stability or integrity of the production environment.
Detail Process: 6.1, Change Standards and Procedures
Set up formal change management procedures to handle in a standardized manner all requests (including maintenance and patches) for changes to applications, procedures, processes, system and service parameters, and …
Detail Process: 6.2, Impact Assessment, Prioritization and Authorization
Ensure that all requests for change are assessed in a structured way for impacts on the operational system and its functionality. This assessment should include categorization and prioritization of changes. Prior to migration to production, changes are authorized by the appropriate stakeholder.
Detail Process: 6.3, Emergency Changes
Establish a process for defining, raising, assessing and authorizing emergency changes that do not follow the established change process. Documentation and testing should be performed, possibly after implementation of the …
Detail Process: 6.4, Change Status Tracking and Reporting
Establish a tracking and reporting system for keeping change requestors and relevant stakeholders up to date about the status of the change to applications, procedures, processes, system and service parameters, and the underlying …
Detail Process: 6.5, Change Closure and Documentation
Whenever system changes are implemented, update the associated system and user documentation and procedures accordingly. Establish a review process to ensure complete implementation of changes.

Areas of IT Service Management that input or output to this process might also include:

Domain: Acquire and Implement    AI7 Process: Install and Accredit Solutions and Changes
New systems need to be made operational once development is complete. This requires proper testing in a dedicated environment with relevant test data, definition of rollout and migration instructions, release planning and actual promotion to production, and a post-implementation review. This assures that operational systems are in line with the agreed expectations and outcomes.
(AI7 controls are fully detailed in the proceding section, Software Development.)

Domain: Deliver and Support DS10 Process: Manage Problems
Detail Process: 10.2   Problem Tracking and Resolution
The problem management system should provide for adequate audit trail facilities that allow tracking, analyzing and determining the root cause of all reported problems considering:
• All associated configuration items
• Outstanding problems and incidents
• Known and suspected errors
Identify and initiate sustainable solutions addressing the root cause, raising change requests via the established change management process. Throughout the resolution process, problem management should obtain regular reports from change management on progress in resolving problems and errors. Problem management should monitor the continuing impact of problems and known errors on user services. In the event that this impact becomes severe, problem management should escalate the problem, perhaps referring it to an appropriate board to increase the priority of the request for change (RFC) or to implement an urgent change as appropriate. The progress of problem resolution should be monitored against SLAs.
Detail Process: 10.4   Integration of Change, Configuration and Problem Management
To ensure effective management of problems and incidents, integrate the related processes of change, configuration and problem management. Monitor how much effort is applied to firefighting rather than enabling business improvements and, where necessary, improve these processes to minimize problems.
Detail Process: 13.2   Job Scheduling
Organize the scheduling of jobs, processes and tasks into the most efficient sequence, maximizing throughput and utilization to meet business requirements. The initial schedules as well as changes to these schedules should be authorized. Procedures should be in place to identify, investigate and approve departures from standard job schedules.

Domain: Deliver and Support: DS4 Process: Ensure Continuous Service
Detail Process: 4.4     Maintenance of the IT Continuity Plan
Encourage IT management to define and execute change control procedures to ensure that the IT continuity plan is kept up to date and continually reflects actual business requirements. It is essential that changes in procedures and responsibilities be communicated clearly and in a timely manner.

Domain: Deliver and Support:  DS8 Process: Manage Service Desk and Incidents
Detail Process: 8.2     Registration of Customer Queries
Establish a function and system to allow logging and tracking of calls, incidents, service requests and information needs. It should work closely with such processes as incident management, problem management, change management, capacity management and availability management. Incidents should be classified according to a business and service priority and routed to the appropriate problem management team, and customers kept informed of the status of their queries.

Domain: Deliver and Support DS9 Process: Manage the Configuration
Detail Process: 9.1     Configuration Repository and Baseline
Establish a central repository to contain all relevant information on configuration items. This repository includes hardware, application software, middleware, parameters, documentation, procedures and tools for operating, accessing and using the systems and services. Relevant information to consider is naming, version numbers and licensing details. A baseline of configuration items should be kept for every system and service as a checkpoint to which to return after changes.
Detail Process: 9.2     Identification and Maintenance of Configuration Items
Put procedures in place to:
• Identify configuration items and their attributes
• Record new, modified and deleted configuration items
• Identify and maintain the relationships among configuration items in the configuration repository
• Update existing configuration items into the configuration repository
• Prevent the inclusion of unauthorized software
These procedures should provide proper authorization and logging of all actions on the configuration repository and be properly integrated with change management and problem management procedures.

Domain: Plan and Organize PO10  Process: Manage Projects
Detail Process: 10.11  Project Change Control
Establish a change control system for each project, so all changes to the project baseline (e.g., cost, schedule, scope and quality) are appropriately reviewed, approved and incorporated into the integrated project plan in line with the program and project governance framework.

The change management process might be characterized as in the following diagram:

Change Management touches the entire enterprise

.

Controls Specific to Software Development and Product Development Lifecycle

Teams preparing their annual audit work papers (controls evidence) will often ask, “What about SDLC related artifacts like VSS/ CVS, test events and source code libraries?  Software Development work products have particular control requirements satisfied through the appropriate use of tools such as workflow and event management, integrated development environments or IDE’s and storage libraries for the purpose of source control.  Unlike controls designed to enforce and manage the policy of production environments however, software development must allow for both creative genius and water tight tests over what can sometimes be bleeding edge or “non conforming” code.  The requirements aligned to creating and modifying today’s business solutions must often support multiple libraries of code  involving issues integration with environments best described as moving targets, highly complex data requirements and greater third party dependencies than could have ever been anticipated. 

The term lifecycle can be taken to represent the collection of agreed steps to control development, modification and distribution of code.  While Change and Configuration Management denote separate entities exerting policy over standards for the production environment, the design of these standards and all efforts between these points can be characterized as the world of software development and code.  “Install and Accredit Solutions and Changes is the high level functional area that captures the greatest number of features representing the activities related to SDLC or Release Management.  AI7 states:

“New systems need to be made operational once development is complete. This requires proper testing in a dedicated environment with relevant test data, definition of rollout and migration instructions, release planning and actual promotion to production, and a post-implementation review. This assures that operational systems are in line with the agreed expectations and outcomes.”

“Install and Accredit Solutions and Changes” include inputs and outputs to configuration, project, change, maintenance and acquisition programs.  With handoffs based in triggers, performance goals, measurements and business based criteria, documented consensus and tested results; evidence of their implementation is best suited to automated systems.

As companies comply with regulatory and industry requirements, system based records showing evidence of these controls gain even greater importance.  The following are high level definitions of the control processes found within Acquisition and Implementation process, Install and Accredit Solutions and Changes, or “AI7”.

Training (7.1)
Train the staff of the affected user departments and the operations group of the IT function in accordance with the defined training and implementation plan and associated materials, as part of every information systems development, implementation or modification project.
Test Plan (7.2)
Establish a test plan and obtain approval from relevant parties. The test plan is based on organization wide standards and defines roles, responsibilities and success criteria. The plan considers test preparation (including site preparation), training requirements, installation or update of a defined test environment, planning/performing/documenting/retaining test cases, error handling and correction, and formal approval. Based on assessment of the risk of system failure and faults on implementation, the plan should include requirements for performance, stress, usability, pilot and security testing.
Implementation Plan (7.3)
Establish an implementation plan and obtain approval from relevant parties. The plan defines release design, build of release packages, rollout procedures/installation, incident handling, distribution controls (including tools), storage of software, and review of the release and documentation of changes. The plan should also include fallback/back out arrangements.
Test Environment (7.4)
Establish a separate test environment for testing. This environment should reflect the future operations environment (e.g., similar security, internal controls and workloads) to enable sound testing. Procedures should be in place to ensure that the data used in the test environment are representative of the data (sanitized where needed) that will eventually be used in the production environment. Provide adequate measures to prevent disclosure of sensitive test data. The documented results of testing should be retained.
System and Data Conversion (7.5)
Ensure that the organization's development methods provides for all development, implementation or modification projects, that all necessary elements such as hardware, software, transaction data, master files, backups and archives, interfaces with other systems, procedures, system documentation, etc., be converted from the old system to the new according to a pre-established plan. An audit trail of pre- and post-conversion results should be developed and maintained. A detailed verification of the initial processing of the new system should be performed by the system owners to confirm a successful transition.
Testing of Changes (7.6)
Ensure that changes are tested in accordance with the defined acceptance plan and based on an impact and resource assessment that includes performance sizing in a separate test environment by an independent (from builders) test group before use in the regular operational environment begins. Parallel or pilot testing should be considered as part of the plan. The security controls should be tested and evaluated prior to deployment, so the effectiveness of security can be certified. Fallback/ backout plans should also be developed and tested prior to promotion of the change to production.
Final Acceptance Test (7.7)
Ensure that procedures provide for, as part of the final acceptance or quality assurance testing of new or modified information systems, a formal evaluation and approval of the test results by management of the affected user department(s) and the IT function. The tests should cover all components of the information system (e.g., application software, facilities, technology and user procedures) and ensure that the information security requirements are met by all components. The test data should be saved for audit trail purposes and for future testing.
Promotion to Production (7.8)
Implement formal procedures to control the handover of the system from development to testing to operations in line with the implementation plan. Management should require that system owner authorization be obtained before a new system is moved into production and that, before the old system is discontinued, the new system has successfully operated through all daily, monthly, quarterly and year-end production cycles.
Software Release (7.9)
Ensure that the release of software is governed by formal procedures ensuring sign-off, packaging, regression testing, distribution, handover, status tracking, backout procedures and user notification.
System Distribution (7.10)
Establish control procedures to ensure timely and correct distribution and update of approved configuration items. This involves integrity controls; segregation of duties among those who build, test and operate; and adequate audit trails of all actions.
Recording and Tracking of Changes (7.11)
Automate the system used to monitor changes to application systems to support the recording and tracking of changes made to applications, procedures, processes, system and service parameters, and the underlying platforms.
Post-implementation Review (7.12)
Establish procedures in line with the enterprise development and change standards that require a post-implementation review of the operational information system to assess and report on whether the change met customer requirements and delivered the benefits envisioned in the most cost-effective manner.
(Additional information regarding CobiT® can be viewed at http://www.isaca.org)

While supporting programs, such as Steering Committee and overall Project / Program Management, Training and Quality Assurance have their own requirements for Process documentation, controls for Software Development cannot be achieved without cooperation between these groups.  Flow of events supporting SDLC maturity include production release acceptance.  The acceptance of any configuration could involve any business unit, engineering, third party service or client approval.  Artifacts of these procedures can include process profiles, detailed workflow diagrams, committee presentations or on line help files, but the true nature of software development controls evidence is demonstrated through the appropriate use and capture of controls over the policies and procedures for the development and release of all associated software and code.  Modern forms of evidence generally must:

• Demonstrate an existing system context in the form of functional, transaction process and infrastructure diagrams (e.g., high-level business process flow schema).
• Quantify by class, a hierarchical data flow/control flow decomposition.
• Include control transformations.
• Allow for mini-specifications.
• Maintain data dictionaries.
• Consider all external events-inputs from external environment.
• Track by change any and every single transformation data flow diagrams with extreme emphasis towards data input, transformation, verification, validation and output controls. 

 

Consider the seven Information Criteria as represented in review of IT Governance Controls by ISACA, effectiveness, efficiency, confidentiality, integrity, availability, compliance and reliability.  In order for any process to meet these criteria, there will be dependencies on programs outside of Development as an organizational unit.

Figure 10: Information Criteria

Regardless of domain, process outputs are reviewed in the context of their effectiveness, efficiency, confidentiality, integrity, availability, compliance and reliability.  Given the scenario of software development programs, information criteria might be applied to elements such as the following:

• Code Libraries
• Restore by code revision and secure retrieval process
• Meeting Minutes as visible in tracking systems
• Risk Evaluation Definition and Risk Review
• Anomaly Measures and Reporting
• Enhancement Request Tracking
• Quality and Test Tracking
• Test Plans
• Project Milestone Tracking
• Data Dictionary
• Defect, Enhancement or Error Tracking
• Production Acceptance guidelines
• Post Implementation Review
• Customer Satisfaction Ratings
• Service Level and Operating Level Requirements

Version control software, or tools that capture a roll back state, are sometimes confused with full scale software development tracking applications.  Online Programming Facilities (integrated Development Environment-lDE), however, must extend far beyond a snapshot and restoration to previous version of code.  While software projects were once a routine process of delivering code to a single business unit, involving a homogenous development teams using a single platform and familiar technology, today’s projects are infinitely more complex.  Typical projects integrate efforts by dozens of developers in multiple countries, involving three or more business streams, traversing varied platforms and applications, accepting some degree of technological uncertainty and unfamiliarity, and satisfying the requirements of disparate organizations that may not have organic opportunities to seek consensus or even share awareness of each other’s requirements.  Add to this at least one major vendor, market projections for cost and completion and increasingly regulated and complex electronic transaction processes.  Where audit and control are concerned, a code snapshot will not suffice.

Programming tools are not enough to facilitate effective use of structured service delivery.  Highly skilled program teams require systems to enable proper use of best practice, including protection of their own use or misuse as a primary IT resource.  Mature shops leverage an online programming facility as part of an integrated development environment.  This practice alone, however, cannot assure mature product delivery.  Software departments require SDLC products, where modules go beyond use by programmers and include functions to support all members in the path of a Service Application Management.  While an IDE provides programmers ability to code and compile programs interactively with a remote computer, it cannot efficiently and effectively control workflow, to say nothing of risk management. 

In fact, the IDE alone can equally facilitate control weakness in IT, in that anyone with access may modify or delete programming code, as well as compile and store programs (source and object) on a single workstation without prior plan, authority, monitoring or approval.  While affording required reporting, the online facilities if not properly controlled can allow rogue processes to update and retrieve data directly from computer files.  While the development platform implementation is often a business requirement, without proper controls, it is also a control risk.  This is where the analysis of use and process surrounding SDLC become critical to the audit process.

Process Flow Diagram

Steve Covey’s often quoted principle “Begin with the end in mind” applies well to the question of “what do I need to capture during the process flow diagramming process?”  Accurate, versus incomplete requirements are said to represent the single greatest factor in software development success.  Consider that the process of gathering requirements provides many opportunities to communicate attributes needed for successful documentation of software and business controls.  Regardless of the applications used to document requirements, using common terms and controls definitions, such as those found in CobiT® 4.0 will dramatically shorten time spent on software design and control documentation.  The following image is suggested content captured by control objects in a process flow diagram.  Such objects exist in virtually all process modeling tools. 
The effort to apply controls language to the documentation of responsibility is a process driven by people, and best facilitated by current and mature software development and process modeling tools.  The examples are created using Microsoft Visio.

When selecting tools for the alignment of SDLC with regulatory and audit reporting requirements, consider that the product must:

• Easily and clearly represent main components, objectives and user requirements, while identifying areas that require controls
• Provide means for capture, evaluation and rank of the major risks to, and exposures of, the system
• Include how each control is monitored and what ensures controls are implemented such that control owners determine their effectiveness, for example that business users review business requirements, data owners review data access, end users affirm adequacy of training materials and documentation, and so on
• Verify that any software development and change tracking system include:
• Work order and related task assignment history, status, duration and outcome
• Segregated tracking of system and role based access such as console logins and logouts by programmers, ticket updates by end users, program authorization by the business.
• Ensure existence of a reasonable explanation for all program deletions
• Document and assign owners to events based in a system maintenance process:
• business authorization,
• regression testing where code or hardware integration affect end user processing, and
• record of post maintenance test results or any other audit evidence.
• Verify through test and frequent checks, the adequacy of production library security
• Integrate process controls between configuration management, problem management and change management in the process to ensure the integrity of the production resources

The following ISACA charts and flow diagram shows IT Programs in relation to the Software development Lifecycle.  The triangle objects represent audited CobiT® controls, with focus to AI7 and including additional controls from supporting programs.

Process Inputs and Outputs, RACI Chart for AI7 as found in CobiT®  4.0, Copyright of ISACA®

Goals and Metrics as described in CobiT®  4.0, Copyright of ISACA®

Software Development Lifecycle and associated CobiT®  controls