Skip Navigation U.S. Department of Health and Human Services www.hhs.gov
Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner

Evidence-based Technical Assistance for Multistakeholder, Community-based Quality Collaboratives

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to www.ahrq.gov for current information.

Amendment No. 1

AHRQ-09-10008


The purpose of this amendment is to:

  1. Respond to Questions
  2. Provide an Interested Vendor List
  3. Provide updated Section H.9 Security and Privacy Requirements
  4. Include the AHRQ Application and System Development Requirements
  5. Include the Information Technology (IT) Privacy Impact Assessment (PIA) Guide

1. Response to Questions

  1. On p. 16, task 4 requires one in person meeting, but the schedule suggests two meetings annually and the performance evaluation factors on p. 35 states this explicitly. Which number is correct?

    Task 4 on p. 16 refers to Option Years 1-4. For the option years, only one in-person meeting per year is required. For year one of the contract, however, two in-person meetings are required.

  2. Task 5 on p. 17 is equal to performance factor six. And although the workgroup on standard report elements is mentioned repeatedly, it is not discussed in the list of tasks. We don't know whether this task is still a requirement.
  3. Task 5: "Convene workgroup on standard public report elements" is a requirement.

  4. Web site:

    a. Does the Requests for Proposal (RPF) require building a Web site from scratch or just maintaining and updating the current version?

    The requirement is for maintenance of and updating the existing Web site.

    b. If not to be built from scratch what platform is the Web site built on?

    The Web site is built on the Microsoft® Windows® Server platform.

    c. Is there a content management system? What is it?

    The content management system is ArrowClick.

    d. Does AHRQ pay for hosting fees or the contract awardee?

    AHRQ pays for the hosting fee through this contract.

    e. Is the site currently 508 compliant?

    Yes.

    f. Is the current Information Technology (IT) security plan transferable with Web site management?

    The awardee must comply with all of AHRQ's standard security requirements (Section H.9) and perform all the steps necessary for completing and delivering a Certification and Accreditation (C&A) package along with a Systems Security Plan (SSP) and all other associated artifacts before the system goes live. However, if the system is already live from previous contract work, the awardee must have the system certified and accredited within 30 days after contract award and deliver the SSP upon contract award. In this case, the SSP should be available from the previous contract and may be modified and resubmitted.

  5. Who is currently providing the services outlined in the RFP?

    Center for Health Improvement

  6. Does the Quality Alliance Steering Committee (QASC) &/or AHRQ drive content?  Webinar Topics?  Will this be driven by these parties or the awardee in the future?

    AHRQ develops the content for the Learning Network Activities, including Webinars, based on the needs expressed by the Chartered Value Exchange (CVE) communities.  AHRQ will solicit the awardee's input in this development process.

  7. Webinar Series: a. It appears the total number of Webinars is approximately 36 per year (2 QASC, 4 Agency Director, 30 contractor). Is each Webinar to have an honorarium paid?

    For QASC updates, AHRQ Agency Director updates, and for CVEs who are invited to play a role on the remaining Webinars, no honorarium is expected or assumed.  For some of the remaining Webinars, it is assumed that Federal experts will be featured, and no honorarium is expected or assumed for them. For non-Federal experts invited to participate in a Webinar, an honoraria of $1000 is assumed, which will cover the presentation, slide development and planning call participation. For budgeting purposes, assume up to 30 experts at $1000 per expert per year.

  8. What is the financial bid range expected for this contract?

    AHRQ is not providing a bid range.

  9. Are the travel costs for each attendee at the planned face to face conferences to be added to the RFP response or just the planning and travel management?   i.e., plane, train, hotel, daily per diem.

    Yes, all travel costs for meeting attendees should be added to RFP responses.

  10. Web site:

    a. Does AHRQ pay for hosting fees or the contract awardee?

    Go to Question 3.d.

    b. Can the site be hosted in the awardee data center or does it have to be hosted elsewhere?

    The site may be hosted in the awardee's data center as long as it is certified and accredited and complies with all security policies, procedures and guidelines as identified in the National Institute of Standards and Technology (NIST) 800 Series and the Federal Information Security Management Act (FISMA).

    c.  Is there any additional security training requirement for staff working on this project? If so, please outline requirements, time frames and costs.

    This information is provided under Section H.9-2.0 in the RFP, Information Systems Security Training. Security Awareness Training is provided by AHRQ in the form of online training; therefore, no additional costs are associated with this requirement. The requirements stated in the RFP are:

    "AHRQ and HHS policy requires contractors receive security training commensurate with their responsibilities for performing work under the terms and conditions of their contractual agreements.

    The contractor will be responsible for assuring that each contractor employee has completed the Security Awareness Training as required by AHRQ prior to performing any contract work, and on an annual basis thereafter, during the period of performance of the contract. The contractor shall maintain a listing of all individuals who have completed this training and shall submit this listing to the Project Officer."

    d. Please define Privacy Impact Assessment and Maintenance and the ongoing requirements for awardee.

    As described in Section H.9-1.2 of the RFP, "the Contractor shall conduct and maintain a Privacy Impact Assessment (PIA) as defined by Section 208 of the E-Government Act of 2002 and FAR Clause 52-239-1. Periodic reviews shall be conducted to determine if a major change to the system has occurred, and if a PIA update is needed".

    A blank PIA is attached to this message for further clarification. The PIA is completed through close coordination with AHRQ's Senior Official for Privacy (SOP) and the successful offeror.

    e. Is it necessary for the IT security plan to be part of the RFP response or just delivered upon contract award?  

    The IT security plan must be delivered upon contract award.

  11. Content:

    a.  Is the development of presentations, guidance papers, etc developed by AHRQ or the awardee?  If the awardee, can this content also be shared with the general public on other Web sites beyond CVE?

    The awardee will develop content for presentations, papers, etc. under the direction of AHRQ, and AHRQ maintains rights and ownership of all content developed by the awardee.  If the awardee wishes to share this content on Web sites beyond the CVE Learning Network Web site, the awardee must seek permission from AHRQ to share any content developed for the CVEs, and AHRQ may or may not grant approval for this.

  12. Webinar Series:

    a. What is the average expected size of the Webinars? Is there a cap on the number of people for each Webinar?

    As stated in Task 3, assume an average of 100 participants on each Webinar. Our 2008 experience suggests a range of registrants from roughly 60-110, with a range of actual participants from approximately 30-100. There is no cap on the number of people for each Webinar, and contractor needs to assure capacity beyond 100 participants should it be warranted.

    b.  Are Webinars to be recorded and available for future use?

    As stated in Subtask 3.5, Webinars are to be audio recorded.  As stated in Subtask 3.7, audio recordings of Webinars are to be posted on the Web site for future use.

  13. Meeting Services:

    a. Are the travel costs for each attendee at the planned face to face conferences to be added to the RFP response or just the planning and travel management? i.e., plane, train, hotel, daily per diem.

     A ll travel costs for meeting attendees should be added to RFP responses.

  14. Do suppliers need to be compliant with certain human resource regulatory items?  If so, which regulations?

    Offerors should be compliant with all FAR and HHSAR clauses listed in the RFP.  There are no specific human resource regulations listed in the RFP.

  15. Marketing:

    a.  How will the specific deliverables be positioned—e.g., events and Webinars. Co-branded between contract awardee and AHRQ? Just AHRQ?

    All events and Webinar will be solely AHRQ-branded.

    b. Will a brand development effort be needed? 

    No.

    c. Will awardeebe responsible for all the marketing/communication/PR [public relations] for the events and Webinars or will AHRQ share in this task?

    The awardee is responsible for all marketing/communication/PR for events and Webinars, under guidance from AHRQ.

  16. What is AHRQ's expected level of effort for this project?

    The CVE Learning Network (LN) is—and will continue to be—an AHRQ effort.  AHRQ staff will be on the agenda at the meetings and will be involved in the Webinars, if only to make introductory remarks.  AHRQ staff also will oversee LN decisions and implementation. 

  17. Is AHRQ open to a somewhat different management structure than the stated requirement of 50% project director time on site at AHRQ such as the sharing of on-site responsibility, including attendance at AHRQ meetings, by two senior staff?

    AHRQ seeks a single project director, who is on site at AHRQ at least 20 hours per week.

  18. The RFP states that office space and a telephone will be provided for the on-site project director. Will a computer also be provided? Will the project director be expected to maintain an AHRQ E-mail address?

    It is assumed that the contractor will use a contractor laptop. The contractor will not have an AHRQ E-mail address.

  19. How many CVEs does AHRQ expect to add during the coming years?

    AHRQ has no concrete plans, as of this date, to add CVEs.   Should such plans materialize, a contract modification will be needed, e.g., to support addition of new CVE profiles to the Web site, to support travel for additional participants to meetings, etc.

  20. What security and development documentation will be required as part of the maintenance of the Intranet site?

    The awardee's System Development Lifecycle (SDLC) must be capable of producing AHRQ required system deliverables containing the required content as described further in AHRQ's Application and System Development Requirements. It is required that the awardee use the lifecycle phases defined in the AHRQ SDLC framework and obtain AHRQ Project Officer (PO) approval before moving from one phase to another. The awardee must also conform to AHRQ Configuration Management (CM) and change control standards which require appropriate controls for managing software and documentation baselines, changes to software artifacts using an appropriate Investigational Device Exemption (IDE) or version management tool, document change requests and obtaining approval through a formal change control process that requires PO and possible AHRQ IT approval prior to implementation.

  21. Part I, Section A—Solicitation Form and Part IV, Section L—Instructions, Conditions & Notices to Offerors, Subsection L.13.C.g.

    a. Please confirm whether the overall goal for small business participation applicable for this RFP is 19% (as identified in Part I, Section A on page 1 of the RFP) or 20% (as identified in Part IV, Section L.13.C.g on page 74 of the RFP)? 

    The small business goal is 19%. The 20% on page 74 is an error.

    b. Is this to include 3% for service-disabled veteran-owned small businesses (from page 1) or for veteran-owned small businesses generally (from page 74)?

    The 3% is for Service Disabled Veteran Owned Small Businesses.

  22. Section F.3: Deliverable Schedule, Task 4, Subtask 4.1

    a. Task 4, Subtask 4.1 states "first meeting planned for Spring 2009 which will require faster turn around." Please provide clarification as to which month is referred to by "Spring 2009."

    Our target for the meeting is now late May or June 2009. 

  23. For budget purposes, should it be assumed that each in-person meeting will last two full days and that all participants will be involved for the full two days?

    Yes.

  24. For budget purposes, should it be assumed that each Webinar will last 90 minutes and will have no more than 200 participants?

    Webinars generally last 90 minutes, however QASC and Agency Director briefings often are limited to 1 hour. Our 2008 experience suggests a range of registrants from roughly 60-110, with a range of actual participants from approximately 30-100. There is no cap on the number of people for each Webinar, and contractor needs to assure capacity beyond 100 participants should it be warranted.

  25. Accurately budgeting costs for making the Web site 508-compliant would be easier if bidders were allowed to review it. Could AHRQ give potential bidders access to the site for this purpose?

    The Web site is now 508-compliant.  We are not making the site, which is private, available to bidders.  The majority of new materials posted to the Web site that the contractor will need to make 508-compliant will be Powerpoint slide sets (from Webinars, from meeting presentations) and audio recordings.

  26. To better estimate costs of hosting the Web site, could you please indicatein what language the current CVE Web siteis built( e.g., ASP, PHP) and the type of database it is currently using (e.g., MySQL, Oracle)?

    The Web application/Web site was developed using both ASP and ASP.Net technologies.  The languages used include VisualBasic and C#. The underlying database technology is Microsoft SQL 2008.

  27. What is the extent and nature of the work to be performed on the Web site (e.g., minor programming changes vs. the addition of new features)?

    Most of the nature of the work to be performed on the Web site will be the posting of new materials, including slides (from Webinars, from meeting presentations), audio recordings, calendar updates, etc. and other maintenance.  Major programming changes and addition of many new features are not expected.

  28. How will the work of the workgroup on standard report elements relate to the activities of the QASC and its workgroups (e.g., the QASC National-Regional Implementation Workgroup)?

    AHRQ collaborates and coordinates with other similar national efforts at both the Agency leadership and staff levels to minimize duplication of effort.  The call for a Workgroup on Standard Report Elements came from CVE members at the February and October 2008 CVE meetings.  CVE participants will be selected for the Workgroup on Standard Report Elements based on their reporting experiences, expertise, potential contributions and familiarity with similar QASC and NQF initiatives, which will further assist ongoing coordination among these groups.

  29. Should the "Past Performance Questionnaire" and "Contractor Performance Form" be completed by just the last 5 most relevant/related contracts and subcontracts completed during the past 3 years or also by all contracts and subcontracts currently in process?

    The Past Performance Questionnaire which includes the Contractor Performance Form should be completed by as many contractors or subcontracts as necessary for the offeror to show its past performance.  It is more beneficial to the Government to see questionnaires from organizations with contracts that are similar to this procurement however all past performance will be evaluated.

  30. How many past Webinars & and slidedecks must be converted & posted to the Web site?

    None.

Top of Page

2. Interested Vendor List

Delmarva Foundation for Medical Care, Inc.
Easton, MD
POC: Christian E. Jensen

Healthcare Information and Management Systems Society
Arlington, VA
POC: H. Stephen Lieber

Houck & Associates, Inc.
Boulder, CO
POC: Sue Houck

The Lewin Group, Inc.
Falls Church, VA
POC: Sue Bembers

Milliman, Inc.
Seattle, WA
POC: Jeff Wilson

Network for Regional Healthcare Improvement (NRHI)
Pittsburg, PA
POC: Harold D. Miller

Thompson Reuters Healthcare
Santa Barbara, CA
POC: Jon Conklin

Top of Page

3. Updated Section H.9 Security and Privacy Requirements

Please replace the RFP Section H.9 with the following:

H.9 Security and Privacy Requirements

1. Adherence to security and privacy policy. The Contractor shall comply with all Federal and Department of Health and Human Services (HHS) security and privacy guidelines in effect at the time of the award of this contract.  A list of applicable United States (U.S.) laws, Office of Management and Budget (OMB) requirements, HHS policies, standards and guidance, and Federal Government Computer Security guidelines can be located on the Secure One HHS Web site. The Contractor shall perform periodic reviews to ensure compliance with all information security and privacy requirements.  The Contractor shall make all system information and documentation produced in support of the Contract available to the agency and agency auditors upon request.

2. Perimeter defense and notification.  The Contractor shall ensure that the system and the information it contains are secured using appropriate perimeter defense technologies and that these technologies are monitored for anomalous traffic behavior.  The Contractor shall immediately report any unauthorized system access to the agency Project Officer and/or System Owner. 

3. Protection of sensitive information. The Contractor shall ensure that sensitive information is protected by information security and privacy controls commensurate with the risk associated with the potential loss or compromise of this information.  For purposes of this contract, information is sensitive if the loss of confidentiality or integrity could be expected to have a serious, severe or catastrophic adverse effect on organizational operations, organizational assets, or individuals.1 Further, the loss of sensitive information confidentiality or integrity could: (i) cause a significant or severe degradation in mission capability to an extent and duration that the organization is unable to perform its primary functions or the effectiveness of the functions is significantly reduced; (ii) result in significant or major damage to organizational assets; (iii) result in significant or major financial loss; or (iv) result in significant, severe or catastrophic harm to individuals.

Personally identifiable information (PII) is a subset of sensitive information and is defined as data which can potentially be used to identify, locate, or contact an individual, or potentially reveal the activities, characteristics, or other details about a person.2 PII shall receive a level of protection commensurate with the risk associated with the loss or compromise of sensitive information.

4. Sensitive information on public systems.  The Contractor shall ensure that sensitive information is not stored, processed or transmitted on a publicly-available system (via the Internet) without the appropriate controls in place and specific authorization from the AHRQ Chief Information Officer (CIO). 

5. Privacy requirements.  The Contractor shall conduct and maintain a Privacy Impact Assessment (PIA) as defined by Section 208 of the E-Government Act of 2002 and Federal Acquisition Regulation (FAR) Clause 52-239-1, and required by HHS policy.  The PIA shall be completed in accordance with HHS PIA guidance. Periodic reviews shall be conducted to determine if a major change to the system has occurred, and if a PIA update is subsequently required.  If it is determined that an update is necessary, the Contractor shall make the necessary changes to the PIA.

The Contractor shall abide by all requirements of the Privacy Act of 1974 and FAR Clause 52-239-1. Pursuant to those requirements, the Contractor shall create and publish a System of Records Notice (SORN) in the Federal Register when required and shall publish an updated SORN following a major change to the system, as directed by OMB Memorandum (M) 03-22, OMB Guidance for Implementing the Privacy Provisions of the E-Government Act of 2002, or subsequent replacement guidance.

6. System accreditation.  The Contractor shall certify and accredit all systems in conformance with the standards set forth by the Federal Information Security Management Act (FISMA) and National Institute of Standards and Technology (NIST) Special Publication (SP) 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, prior to the system becoming operational. This activity shall be performed in conjunction with the initial development of the system, updated when a major change occurs to the system, and renewed no less than every three years.  All system certification and accreditation (C&A) packages shall be compliant with all Public Law (PL)-107-347, OMB mandates, FIPS [Federal Information Processing Standards], and additional applicable NIST guidance. This guidance includes, but is not limited to FIPS 199, FIPS 200, NIST SP 800-18, NIST SP 800-30, NIST SP 800-37, NIST SP 800-53, NIST SP 800-53A, and NIST SP 800-60. All NIST and FIPS documentation can be found at the NIST Web site.

HHS has created a C&A checklist to facilitate compliance with the OMB-mandated C&A process.  The HHS C&A Checklist will be provided upon contract initiation.  Prior to becoming operational, all systems must receive a signed Authorization to Operate (ATO) issued by the agency Designated Authorization Authority (DAA).  No system will be permitted into the production environment without a valid, signed ATO.

7. Annual requirements. The Contractor shall be responsible for meeting ongoing information security and privacy system requirements.  These include, but are not limited to, performing annual system testing, completing an annual system self-assessment, and supporting quarterly and annual AHRQ FISMA reporting.  Additionally, AHRQ reserves the right to test or review the system security and privacy controls at any time.

8. Security and privacy training.  All Contractors shall receive general awareness training and role-based training, commensurate with the responsibilities required to perform the work articulated in the terms and conditions of the Contract.

The Contractor shall be responsible for ensuring each contractor employee has completed the AHRQ Security Awareness Training as required by the agency prior to performing any contract work or accessing any system, and on an annual basis thereafter, throughout the period of performance of the contract.  The Contractor shall maintain a list of all individuals who have completed this training and shall submit this list to the Project Officer upon request.  As a part of this training, the Contractor shall ensure that all staff read, agree to, and sign the HHS Rules of Behavior.

The Contractor shall ensure that all contractors with significant security responsibilities, as defined by HHS, receive commensurate role-based training.  As stated in the Secure One HHS Memorandum, Role-Based Training (RBT) of Personnel with Significant Security Responsibilities, significant security responsibilities are defined as the responsibilities associated with a given role or position, which, upon execution, could have the potential to adversely impact the security posture of one or more HHS systems.3  The Contractor shall maintain a list of all individuals that possess significant security responsibilities and the subsequent role-based training courses completed, and shall submit this list to the Project Officer upon request. 

9. Electronic communication.  All Contractor staff that have access to and use of HHS electronic mail (E-mail) must identify themselves as contractors on all outgoing E-mail messages, including those sent in reply or forwarded to another user.  The Contractor shall ensure all contractor staff embed an E-mail signature ("AutoSignature") or an electronic business card ("V-card") within each electronic correspondence to automatically display "Contractor" in the signature.

10. Clearances.  The Contractor shall ensure all staff have the required level of security clearance commensurate with the sensitivity of the information being stored, processed, transmitted or otherwise handled by the System or required to perform the work stipulated by the contract.  At the minimum, all Contractor staff shall be subjected to a Public Trust background check and be granted a Public Trust clearance before access to the System or other HHS resources is granted.

11. Non-Disclosure.  The Contractor shall not release, publish, or disclose agency information to unauthorized personnel, and shall protect such information in accordance with the provisions of the following laws and any other pertinent laws and regulations governing the confidentiality of sensitive information:

  • 18 U.S.C. 641 (Criminal Code: Public Money, Property or Records)
  • 18 U.S.C. 1905 (Criminal Code: Disclosure of Confidential Information)
  • PL 96-511 (Paperwork Reduction Act)

The Contractor shall ensure that each contractor employee who may have access to agency information under this contract shall complete and sign the Commitment to Protect Non-Public Information—Contractor Agreement (Non-Disclosure Agreement).  A copy of each signed and witnessed Non-Disclosure Agreement shall be submitted to the Project Officer prior to performing any work under the contract.

12. Mobile device encryption.  The Contractor shall: (a) encrypt all laptop computers, mobile devices and portable media which store or process, or may store or process, sensitive information using FIPS 140-2 compliant encryption technology; (b) verify that encryption products have been validated under the Cryptographic Module Validation Program to confirm compliance with FIPS 140-2; (c) establish key recovery mechanisms to ensure the ability to decrypt and recover sensitive information by authorized personnel; and (d) generate and manage encryption keys securely to prevent unauthorized decryption of information.  For more information, reference the HHS Encryption Standard for Mobile Devices and Portable Media.

13. Desktop Encryption. The Contractor shall: (a) encrypt all desktop computers which store or process, or may store or process, sensitive information using FIPS 140-2 compliant encryption technology; (b) verify that encryption products have been validated under the Cryptographic Module Validation Program to confirm compliance with FIPS 140-2; (c) establish key recovery mechanisms to ensure the ability to decrypt and recover sensitive information by authorized personnel; and (d) generate and manage encryption keys securely to prevent unauthorized decryption of information.  In the case that appropriate compensating controls are implemented to protect sensitive desktop computers, the requirement for encryption may be waived with approval from the AHRQ Chief Information Security Officer (CISO).  For more information, reference the HHS Encryption Standard for Mobile Devices and Portable Media.

14. Minimum security configurations. The Contractor shall certify applications are fully functional, operate as intended, and comply with the HHS Minimum Security Configurations for Operating Systems (currently HHS has minimum configuration standards for Windows 2000 Server, Windows 2000 Professional, Windows 2003 Server, Windows NT, Windows XP, Solaris, HP-UX, Redhat Linux, Oracle, and Cisco IOS).  These standard security configurations shall be provided to the Contractor at the time of contract initiation and upon completion of the required Non-Disclosure Agreements.  Additionally, the Contractor shall adhere to these configurations when developing the system.  As standard configurations may change frequently, the Contractor must ensure applications remain compliant with the most recent set of security configurations. 

Additionally, for desktops and laptops within the system boundary, the Contractor shall comply with the configurations defined in the HHS Federal Desktop Core Configuration (FDCC) standards, which were designed to meet the requirements mandated by OMB.  The FDCC standards will be provided upon contract initiation.  The installation, operation, maintenance, update, and/or patching of software shall not alter the approved HHS Minimum Security Configurations or the HHS FDCC standards.  Applications designed for normal end users shall run in the standard user context without elevated system administration privileges.  Exceptions to the HHS requirements must be documented, accompanied by compensating controls, and approved by the HHS CISO and the AHRQ CISO in advance of implementation.

15. Maintenance.  The Contractor shall ensure that the system, once operational, is properly maintained and monitored, to include immediate response to critical security patches, routine maintenance windows to allow for system updates, and compliance with a defined configuration management process.  All patches and system updates shall be properly tested in a development environment before being implemented in the production environment.

References

1.Policy for Department-wide Information Security

2.HHS IRM Information Security Program Policy

3.HHS Personnel Security/Suitability Handbook  (PDF file; PDF Help)

4.NIST SP 800-18, Rev. 1, Guide for Developing Security Plans for Information Technology Systems

5.NIST SP 800-37, Guide for Security Certification and Accreditation of Federal Information Systems (PDF file; PDF Help

6.NIST SP 800-53, Recommended Security Controls for a Federal Information System

7.NIST SP 800-60, Guide for Mapping Types of Information and Information Systems to Security Categories, Volume I

8. NIST SP 800-60, Guide for Mapping Types of Information and Information Systems to Security Categories, Volume II )

9. NIST SP 800-64, Security Considerations in the Information System Development Life Cycle 

10.FIPS 199, Standards for Security Categorization of Federal Information and Information Systems (PDF file; PDF Help)

11.Federal Information Processing Standards, Minimum Security, Requirements for a Federal Information System (PDF file; PDF Help)

12.Cryptographic Module Validation Program


1. Federal Information Processing Standard (FIPS) 1999, Standards for Security Categorization of Federal Information and Information Systems, February 2004.
2. HHS Rules of Behavior, February 12, 2008.
3. HHS Memorandum, Role-Based Training (RBT) of Personnel with Significant Security Responsibilities, October 3, 2007. 


Top of Page

4. AHRQ Application and System Development Requirements

AHRQ has implemented a Distributed Systems Engineering Lab (DSEL) to support all internal development efforts and provide the facility for housing the software and documentation for all AHRQ sponsored systems and applications, regardless of where the system or application is hosted.

AHRQ uses a System Development Lifecycle (SDLC) framework which is consistent with the HHS Enterprise Lifecycle (EPLC) framework.  This framework is the basis for implementation of the DSEL, conduct of development projects and the Rational Unified Process (RUP)/Capability Maturity Model (CMM) based processes that support its implementation. The SDLC framework provides a disciplined approach which employs the following traditional project phases:

  • Concept
  • Initiation
  • Planning
  • Requirements Analysis
  • Design
  • Development
  • Testing
  • Implementation/Deployment
  • Operations and Maintenance
  • Retirement

The AHRQ SDLC framework is closely aligned with the disciplines defined in the Rational Unified Process (RUP).  The IBM Rational Suite of tools has been adopted by the Agency to provide a standard IT development environment for AHRQ sponsored systems and application development projects. The AHRQ SDLC framework has been enhanced through the use of tailored processes and artifacts based on the RUP methodology.  The documentation deliverables required for all Information Technology (IT) projects are based on specific RUP artifacts identified by ARHQ.  The Rational ClearCase libraries housed within the DSEL provide the repository for housing software and documentation artifacts related to all AHRQ sponsored systems and applications, regardless of where the system or application is hosted.

Contractors are not required to follow the RUP development methodology or use the Rational Suite of tools; however, the Contractor's SDLC must be capable of producing AHRQ required system deliverables containing the required content as described further in the following section.  It is required that the Contractor use the lifecycle phases defined in the AHRQ SDLC framework and obtain PO approval before moving from one phase to another. The contractor must also conform to AHRQ Configuration Management (CM) and change control standards which require appropriate controls for managing software and documentation baselines, changes to software artifacts using an appropriate IDE or version management tool, document change requests and obtaining approval through a formal change control process that requires Project Officer (PO) and possible AHRQ IT approval prior to implementation.

The following table describes the documentation deliverables required for all IT projects and the content required for each deliverable.

Table 1.1—Documentation Deliverables

Deliverable

AHRQ Life Cycle Phase

Formats

Project Initiation Document

Project Initiation

MS Word®

Project Work Plan

Project Planning

MS Project®

System Requirements Document (SRD)

Requirements and Analysis

Rational Requisite Pro, MS Word®

Requirements Traceability Matrix

Requirements and Analysis

Rational Requisite Pro, MS Word®

System Design Document (SDD)

Software Design

Rational Software Modeler, MS Word®,
Rational Software Architect

Test Plan

Testing

MS Word®

Test Scripts

Testing

MS Word®, Rational Test Manager

User Acceptance Testing Report

Implementation

MS Word®

User Guide

Deployment

MS Word®

Operations Manual

Deployment

MS Word®

Version Description Document

Deployment

MS Word®

System Documentation

The Contractor will provide to the Agency system documentation of all proposed hardware, software, security, backup/recovery, and other information technology infrastructure and components and solutions needed to support this project.  The documentation is to be delivered to the Project Officer for review and approval for each release.  This documentation will be provided according to the content standards specified by AHRQ and will be maintained in the Agency's Rational ClearCase Repository as a unique project library created and maintained by the AHRQ CM Manager.  All documentation will be baselined with each system release. In addition, the source code for each production release will be delivered and stored in the same project library as the documentation artifacts. The contractor will be required to update these baselined artifacts for each production release of the system.  Sample documents and templates for the required documentation artifacts are available as guidance.  The following documents as mentioned in Table 1.1, "Documentation Deliverables", are required by AHRQ.

Project Initiation Document

The Project Initiation Document (PID) is intended to be a statement of purpose and scope for initiating a given project and a guide to manage expectations in both process and deliverables throughout the System Development Lifecycle (SDLC).  The PID defines the business case for the project by defining the purpose, the milestones, resources, objectives, costs, risks including mitigation strategies, and the artifacts and IT technologies (architecture) utilized and produced for, and during, the project. The PID serves as the formal funding commitment document approved by the COTR and Stakeholders.  Additionally, the PID must be approved by AHRQ IT management, and in some cases, the AHRQ Information Technology Review Board (ITRB) for technical viability, adherence to Agency Enterprise Architecture (EA); technical standards and formal Project Management requirements as derived from Departmental standards and accepted Project Management Institute (PMI) Project Management Body of Knowledge (PMBOK) standards.  In the case of external development contracts, the PID is satisfied by the formal proposal submitted by the vendor and accepted by AHRQ. 

Project Work Plan

The System Project Plan or Project Work Plan (PWP) provides a method to assign and track the project resources, hours and specific deliverables. This plan provides the detailed Work Breakdown Structure (WBS) and resource loading that can be used to identify project costs and is intended for the project manager to track the schedule and cost of a project, including development of Earned Value Management (EVM) measures. The PWP is delineated by the phases of the project which include Project Initiation, Generation of the PWP Schedule, Requirements Gathering, System Design, System Development, System Testing including User Acceptance, System Deployment and System Support and production of project deliverables which require COTR or Stakeholder acceptance and signoff to continue project tasks identified in the PWP. 

System Requirements Document

The System Requirements Document (SRD) contains the system requirements, use cases and supplementary specifications that provide the basis for design and development of the system.  The following information is provided for each requirement identified in the document:

  • Requirement ID, Name and Title
  • Requirement Description
  • Software Release Version
  • Use Case Model
  • Use Case Specifications
  • Supplementary Specifications

A text-based Functional Requirements Document may be provided instead of a Use Case Model, Use Case Specifications, and Supplementary Specifications.

Requirements Traceability Matrix

The Requirements Traceability Matrix (RTM) associates requirements with the work products that satisfy them.  This matrix is created at the beginning of a project's lifecycle to trace the requirements from identification through testing.  The project elements are traced as they relate to other project elements, especially those related to requirements.

The purpose of establishing traceability is to help understand the sources of requirements, manage the scope of the project, manage changes to requirements; assess the project impact of a change in a requirement; and verify that all requirements of the system are fulfilled by the implementation. 

The following values are required for the traceability matrix:

  • Requirement ID and Title.
  • The version of the system in which the requirement will be implemented.
  • The Use Case to which the requirement can be traced.
  • The version of the design document in which the requirement is implemented.
  • The ID of the test script in which the requirement is tested.
  • The version number of the source code in which the requirement is implemented.

The figure below shows a sample of the data traced through a project's life cycle.

Sample of the data traced through a project's life cycle. Column 1 lists requirements for the project; subsequent columns are used to note Version, Trace to UC [Use Case], Trace to Design, Trace to Test, CR, and Status.

System Design Document

The System Design Document (SDD) details the design and implementation of all custom software features of the system. The design descriptions must include use cases that detail the interaction which occurs between a user and the system. 

The document describes the general nature of the system, and describes the architecturally significant parts of the design model, such as its decomposition into subsystems and packages.  For each significant package, a section of the document should detail its decomposition into classes and class utilities.  Architecturally significant classes should be introduced and a description of their responsibilities should accompany the introduction.  Any significant relationships, operations, and attributes should be detailed in this document.

The document should be organized by use case, so that it provides traceability back to the initial requirements. The document must also contain a description of the database model and data elements used to support the application. This data can be referenced to an appropriately maintained Entity Relationship Diagram (ERD) and data definitions which conform to CM standards and are appropriately maintained in the Rational CM Libraries.

Test Plan

The purpose of the Test Plan is to define the approach for testing a particular application or system. The Test Plan is a high level description of the testing process which will be performed. The Test Plan outlines the types of testing to be performed, the requirements to be tested, the test environment, testing tools, pass/fail criteria and a risk assessment. At a minimum the document should contain the following:

A. Test Description
  • A general overview of the plan for testing the entire system.
  • Test objectives for all testing levels (e.g., module, unit).
  • Scope and guiding principles for the testing effort.
  • A policy for resolving conflicts that arise during the testing process.
B. Acceptance Criteria
  • The criteria agreed upon with the customer for acceptance of the software.
C. Approach
  • How each major group of software features will be adequately tested.
  • Major testing activities, techniques, and testing tools.
  • Test Environment—Hardware, Network, Software and Test Database.
D. Tasks
  • The individual tasks that must be performed.
  • The individual or organization responsible for each task.
E. Schedule, Resources & Milestones

Test Scripts

The Test Scripts define testing scenarios completed for an application. Each scenario details the steps to be performed, expected results and pass/fail criteria.  At a minimum the document should contain the following:

  • Test Script Identifier
  • Test Description
  • Test Objective
  • Test Environment/Setup including any required data such as Login credentials, etc.
  • Mapping to specific requirements and design elements contained in the SRD and SDD
  • Step sequences and actions
  • Expected Results
  • Pass/Fail Criteria
  • Actual Results
  • Comments

User Acceptance Test Report

The User Acceptance Testing (UAT) Report should include a summary of the testing environment (hardware, software, tools, participant list, etc.) and procedures used to demonstrate and obtain stakeholder approval of the application or system prior to production deployment. The UAT Report should contain a mapping to the SRD and SDD items included in the release as well as an exception list or identified change requests that were generated as the result of testing.

User Guide

The User Guide is completed prior to production.  The User Guide is a "How To" manual which navigates the user in detail through the use of the application.  This document usually contains system screen shots and provides step by step instructions for completing tasks and activities.  It is written on a business level with the needs of the user in mind.  At a minimum the document should contain the following content.

  • Introduction
  • Summary of the application
  • Glossary (Definitions/Acronyms)
  • Procedures (step-by-step instructions on how to use the system)
  • Troubleshooting tips

Operations Manual

The Operations Manual provides guidance and defines procedures related to the operational implementation of the system. At a minimum, the document should contain the following:

  • System Overview
  • Statement of acceptable use of the system and information
  • Hardware and software descriptions
  • Interfaces with other Systems and Databases
  • Access and authentication requirements
  • System Configuration and Administration Procedures
  • Security procedures including virus protection
  • Incident Reporting and Handling
  • System Startup and Recovery Procedures
  • Change Management Procedures

Version Description Document

The Version Description Document (VDD) identifies and describes the general release information, and inventory of software released (Bill of Materials), for a specific application, including prototype iterations. The document should include the following sections listed below:

  • Introduction—Describes the objective of the document, defines the release identification and provides contact information.
  • General Release Information—Provide information about the specific release, including any interfaces and dependencies.
  • Installation Instructions—Describes the steps required to install the software.
  • Version Description—Provides an inventory of List Objects and Module Types such as: class files, SQL Scripts, HTML files, DTD and XML files.
  • Recovery Instructions—Describe the steps required to reconstruct the release from the product baselines, established in the configuration management library.

Web Product and Web Site Development Guidelines

The following list highlights basic issues that need to be addressed when developing Web tools or sites under contract that will be publicly available when launched to ensure deliverables are on target, in compliance with legal and policy requirements, and do not require expensive rework to meet Federal and Department of Health and Human Services requirements for information resources. 

Guidelines for Web-Based Products

Retrofitting Web-based products after the fact is highly undesirable because it adds time and costs to the process of making these products publicly available. All products that are developed with the intent of being posted on the AHRQ Web site should meet the following minimum requirements:

Titles of Products

Coordinate with your project officer on the titles of your products. They need to be concise and relevant to the purpose of the project, but cannot include the name of the contractor or grantee as the performing organization as part of the title. Report titles should be no more than 10-words maximum and Web-based tools should be no more than 5-words maximum (make every word count—eliminate initial articles such as "The" or "A"). Titles need to be distinct enough to differentiate among similar sounding products.

Quality Control/Editorial Review

This involves checking for spelling and grammar mistakes, formatting issues, general consistency, and style.  This should be done by the AHRQ grantee or contractor prior to submission of the final product for posting on the AHRQ Web site. Federal resources follow the GPO Style Manual which is available electronically at: http://www.gpoaccess.gov/stylemanual/browse.html

Accessibility

As an agency of the Federal Government, AHRQ must ensure that anything that is posted on our Web site is in compliance with requirements for information resources under Section 508 of the Rehabilitation Act.  Also, federally funded resources need to be generally available to users in multiple formats to ensure that we are not forcing a particular platform, operational system, or proprietary software package on users.

Intellectual Property Rights

Before we can post a product on the AHRQ Web site, we must have a written explanation of the following four questions:

  • Who retains the copyright?
  • Who has licenses for what purposes and uses?
  • What are the constraints imposed?
  • Who grants permission for further use or adoption?
Usability

Web resources should include usability testing, evaluation, and modification as an integral and recurring part of the development effort to ensure they are effective for the electronic business processes they are designed to facilitate. A set of Research-Based Web Design and Usability Guidelines that should be consulted are available at: http://www.usability.gov/guidelines/index.html

Beta testing prior to release is desirable, evaluating the product against usability heuristics. As feedback is received and products are updated, the revisions will need to be designated by version number and date of release.

Privacy Act Protections

Web resources are subject to the Privacy Act and this can impact both the development of Web-based tools and the users of those tools. Persistent cookies should not be programmed into the functionality of a Web-based tool, although session cookies are allowed. Registration for use cannot be requested if this would involve collection of individual identifiers from the users. Although exemptions to both rules can be sought, this involves a strong justification and several levels of review for approval through the U.S. Department of Health and Human Services (HHS).

Guidelines for Web Sites

Web sites being supported through contracts are considered Federal information resources and as such are required to be in compliance with laws, policies, and directives that affect such resources.

This includes content management and information categorization, including standard metadata, under the E-Gov Act requirements and Office of Management and Budget issuances to Federal agencies on IT resources.

For recommendations and guidance on requirements and best practices, go to: http://www.firstgov.gov/webcontent/reqs_bestpractices/best_practices.shtml

Clearance

Web resources require clearance by HHS—including justification against a set of criteria. Publications cleared for printing are cleared for Web uploading at the same time. Web resources must comply with the numerous laws and directives that affect federally funded electronic information resources. Web content loaded on a site by contractors must be appropriate and follow all laws and directives. AHRQ Offices and Centers must coordinate initial review through AHRQ's Office of Communications and Knowledge Transfer (OCKT) before launch, and OCKT will coordinate departmental clearance.

Domain Names

All domain names for any Web resource funded in whole or in part by Federal funds must be registered as .gov through HHS with the General Services Administration (GSA). Although other domains, such as .org, .net, .edu, .com may also be reserved by the Agency, the .gov domain must be registered and that domain name will need to be indexed by FirstGov, the GSA portal to government-funded resources. The FirstGov link is then required on the home page of the site. Coordinate with OCKT on domain name requests.

Editorial Review

All content for upload needs to be reviewed to ensure consistency and compliance with best practices and established style and conventions. At a minimum, the copy needs to be production edited to ensure there are no typos and the GPO Style Manual is followed for punctuation, spelling, use of numerals, abbreviations, etc. Do not use unexplained acronyms; they need to be spelled out on first reference in any document or file. There should not be anything marked DRAFT on a public site. Once the materials are uploaded, they are published and considered in the "public domain." Do not use placeholders for content that does not exist. Government funded sites should not have anything designated "under construction." A process needs to be established for regular review of content and updating. Additional materials need to undergo editorial review and be approved before uploading. The GPO Style Guide is available electronically as a reference at: http://www.gpoaccess.gov/stylemanual/browse.html.

Accessibility

Under the Rehabilitation Act, Federal agencies have an obligation to provide equal access for the disabled to their information and services. Requirements are specified in section 504 for individual accommodation and more recently in section 508 for electronic and information technology, which includes Web sites and multimedia products. Equivalent alternatives are required for auditory and visual information, such as providing alternative descriptive text for images for the blind and providing captions for audio-video files for the deaf. Written transcripts are required for all streaming audio. PDF files can be offered in conjunction with accessible files, such as HTML versions, but avoid uploading PDF-only versions of documents unless they are fully accessible PDF formats. OCKT has software used to evaluate Web sites and can provide a report on any accessibility violations that would need to be addressed before launch. Specific requirements are available at: http://www.section508.gov

Privacy

A privacy policy notice must be prominently displayed, and the Web site host has to follow it. A machine-readable format (P3P) of the privacy policy notice must also be uploaded to the site. A Privacy Impact Assessment is conducted to determine what kind of personal information is contained within a system, what is done with that information, and how that information is protected.  Sites are periodically audited to ensure that they observe their stated privacy policy. A Privacy Act System notice may need to be prepared and published for users to register on a site if the registrations represent a group of records, under the control of the Agency (or a contractor), that can be retrieved by personal identifier. This notice must go through several levels of review—including the Office of General Counsel—and be published in the Federal Register. Persistent cookies cannot be used on Federal sites unless the Secretary of HHS grants an exemption, and this involves a strong justification and review process.

Web Site Mailbox

Every Web site must provide full contact information for the sponsor and have a Contact Us link for submission of comments or questions as a customer feedback mechanism. Web site E-mail is subject to the same privacy and records management issues that affect the overall Web site as well as departmental standards for handling inquiries and customer feedback. Each Web site must provide relevant Frequently Asked Questions that are included in the customer relationship management system used to handle AHRQ Web site inquiries.

Records Management

All content on the site and E-mail generated by the site must be archived electronically and handled according to records retention schedules and disposition authorities as established with the National Archives and Record Administration. This requirement also affects Web site log files and statistical reporting on Web site usage. For guidance on requirements, go to:

http://www.archives.gov/records-mgmt/policy/managing-web-records-index.html

Information Collection Budget

If a Web site is used to collect information from users, such as for surveys, evaluations, or beta testing feedback, then the Office of Management and Budget must first approve the burden hours for such an effort for this collection. A notice must be posted on the Web site at the point of collection with the OMB approval number and a statement on the process of collection. 

Intellectual Property

Copyright and trademark protections need to be observed on Web sites. Permissions for use must be granted for any copyrighted information included and registered trademarks need to be reflected in copy. Any copyright or trademark constraints related to materials uploaded to a site must be specified for users. Public domain does not extend outside the borders of the United States. Therefore, foreign countries must request specific permission for use. Given the global nature of the Internet, citation as to source is a critical issue.

Linking

External links constitute an implied endorsement and create a business advantage for the linked sites. OMB requires Agencies to do a risk assessment of external links, and potential links need to be assessed against the HHS and AHRQ linking policies and criteria. If a site deviates from these policies, then the specific review and selection criteria must be justified and posted on the Web site for full disclosure. Outside Web resources may link to Agency resources providing that the link is not displayed in any way that would imply an endorsement by the Agency of a specific commercial product or service.

Electronic FOIA

The Agency is required by law to have an electronic Freedom of Information Act (FOIA) reading room and to provide materials that can be requested under FOIA in electronic form, if so requested. HHS requires that any Web resource funded by the Agency provide a link to the AHRQ Freedom of Information Act page on the main AHRQ Web site.

Usability

Web resources should include usability testing, evaluation, and modification as an integral and recurring part of the development effort to ensure they are effective for the electronic business processes they are supposed to facilitate. Go to http://www.usability.gov as a reference for best practices in initial development or redesign of Web resources.

Web Sponsor Identity

AHRQ has uniform principles to identify AHRQ as the primary sponsor of AHRQ-related Web sites. These principles reflect HHS best practices for a consistent look and feel of Web resources, reinforce credibility, and support HHS and Agency branding efforts. The four specific principles that should be consistent across all AHRQ-funded Web sites are:

  • Web site URL name: The name of a Web site should always contain AHRQ in the URL. A Web resource should either be a folder on the main AHRQ Web site (http://www.ahrq.gov/chiri) or a third-level domain of the Web site (http://www.webmm.ahrq.gov).
  • Title of Web site project: AHRQ's name should be part of the formal title and appear at the beginning of the Web site's project name when referenced in print or promotional materials. For example: AHRQ's Web Morbidity and Mortality online journal.
  • HHS and AHRQ logos: The HHS and AHRQ logos should be featured prominently on the Web site and in materials that are used to market that Web site.
  • Web site home page format: The Web site home page should have common design and navigation elements with the HHS Portal and the AHRQ Web site so that all Web sites look as though they belong to the Department and AHRQ Web family. All AHRQ domain sites must include a standard banner and footer that are branded for Web resources. Technical specifications and templates for developers to consult when designing Web resources are provided by the AHRQ Web Manager.

Top of Page

The Information Technology (IT) Privacy Impact Assessment (PIA) Guide

Select to access the Privacy Impact Assessment (PIA) Guide (PDF file, 705 KB; PDF Help).

Top of Page

Current as of January 2009

The information on this page is archived and provided for reference purposes only.

AHRQ Advancing Excellence in Health Care