Skip Navigation U.S. Department of Health and Human Services www.hhs.gov
Agency for Healthcare Research Quality www.ahrq.gov
Archive print banner

The John M. Eisenberg Center for Clinical Decisions and Communications Sciences: Request for Proposal (Continued)

Select to download RFP documents
Return to Contents

SECTION L - INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

L.1 SOLICITATION PROVISIONS INCORPORATED BY REFERENCE (FEB 1998) (FAR 52.252-1)

This solicitation incorporates the following solicitation provisions by reference, with the same force and effect as if they were given in full text. The full text of a clause may be assessed electronically at this address: http://www.arnet.gov/far/.

  1. Federal Acquisition Regulation (FAR) (48 CFR Chapter 1) Solicitation Provisions
    1. 52.215-16 Facilities Capital Cost of Money (OCT 1997)
    2. 52.215-20 Requirements for Cost or Pricing Data or Information Other Than Cost or Pricing Data (OCT 1997)

L.2 DATA UNIVERSAL NUMBERING (DUNS) (OCT 2003) (FAR 52.204-6)

  1. The offeror shall enter, in the block with its name and address on the cover page of its offer, the annotation "DUNS" or "DUNS+4" followed by the DUNS number or "DUNS+4" that identifies the offeror's name and address exactly as stated in the offer. The DUNS number is a nine-digit number assigned by Dun and Bradstreet Information Services. The DUNS+4 is the DUNS number plus a 4-character suffix that may be assigned at the discretion of the offeror to establish additional CCR records for identifying alternative Electronic Funds Transfer (EFT) accounts (see Subpart 32.11) for the same parent concern.
  2. If the offeror does not have a DUNS number, it should contact Dun and Bradstreet directly to obtain one.
    1. An offeror may obtain a DUNS number—
      1. If located within the United States, by calling Dun and Bradstreet at 1-866-705-5711 or via the Internet at http://www.dnb.com; or
      2. If located outside the United States, by contacting the local Dun and Bradstreet office.
    2. The offeror should be prepared to provide the following information:
      1. Company legal business name.
      2. Tradestyle, doing business, or other name by which your entity is commonly recognized.
      3. Company physical street address, city, state and Zip Code.
      4. Company mailing address, city, state and Zip Code (if separate from physical).
      5. Company telephone number.
      6. Date the company was started.
      7. Number of employees at your location.
      8. Chief executive officer/ key manager.
      9. Line of business (industry)
      10. Company Headquarters name and address (reporting relationship within your entity).

(End of provision)

L.3 INSTRUCTIONS TO OFFERORS - COMPETITIVE ACQUISITION (MAY 2001) ALTERNATE I (JAN 2004)(FAR 52.215-1)

  1. Definitions. As used in this provision –

    "Discussions" are negotiations that occur after establishment of the competitive range that may, at the Contracting Officer's discretion, result in the offeror being allowed to revise its proposal.

    "In writing," "writing," or "written" means any worded or numbered expression that can be read, reproduced, and later communicated, and includes electronically transmitted and stored information.

    "Proposal modification" is a change made to a proposal before the solicitation's closing date and time, or made in response to an amendment, or made to correct a mistake at any time before award.

    "Proposal revision" is a change to a proposal made after the solicitation closing date, at the request of or as allowed by a Contracting Officer as the result of negotiations.

    "Time," if stated as a number of days, is calculated using calendar days, unless otherwise specified, and will include Saturdays, Sundays, and legal holidays. However, if the last day falls on a Saturday, Sunday or legal holiday, then the period shall include the next working day.

  2. Amendments to solicitations. If this solicitation is amended, all terms and conditions that are not amended remain unchanged. Offerors shall acknowledge receipt of any amendment to this solicitation by the date and time specified in the amendment(s).

  3. Submission, modification, revision, and withdrawal of proposals.

    1. Unless other methods (e.g., electronic commerce or facsimile) are permitted in the solicitation, proposals and modifications to proposals shall be submitted in paper media in sealed envelopes or packages (i) addressed to the office specified in the solicitation, and (ii) showing the time and date specified for receipt, the solicitation number, and the name and address of the offeror. Offerors using commercial carriers should ensure that the proposal is marked on the outermost wrapper with the information in paragraphs (c)(1)(i) and (c)(1)(ii) of this provision.

    2. The first page of the proposal must show—
      1. The solicitation number;
      2. The name, address, and telephone and facsimile numbers of the offeror (and electronic address if available);
      3. A statement specifying the extent of agreement with all terms, conditions, and provisions included in the solicitation and agreement to furnish any or all items upon which prices are offered at the price set opposite each item;
      4. Names, titles, and telephone and facsimile numbers (and electronic addresses if available) of persons authorized to negotiate on the offeror's behalf with the Government in connection with this solicitation; and
      5. Name, title, and signature of person authorized to sign the proposal. Proposals signed by an agent shall be accompanied by evidence of that agent's authority, unless that evidence has been previously furnished to the issuing office.
    3. Submissions, modification, revision, and withdrawal of proposals.
      1. Offerors are responsible for submitting proposals, and any modification or revisions, so as to reach the Government office designated in the solicitation by the time specified in the solicitation. If no time is specified in the solicitation, the time for receipt is 4:30 p.m., local time, for the designated Government office on the date that proposal or revision is due.
        1. Any proposal, modification, or revision received at the Government office designated in the solicitation after the exact time specified for receipt of offers is "late" and will not be considered unless it is received before award is made, the Contracting Officer determines that accepting the late offer would not unduly delay the acquisition; and -
          1. If it was transmitted through an electronic commerce method authorized by the solicitation, it was received at the initial point of entry to the Government infrastructure not later than 5:00 p.m. one working day prior to the date specified for receipt of proposals; or
          2. There is acceptable evidence to establish that it was received at the Government installation designated for receipt of offers and was under the Government's control prior to the time set for receipt of offers; or
          3. It is the only proposal received.
        2. However, a late modification of an otherwise successful proposal that makes its terms more favorable to the Government, will be considered at any time it is received and may be accepted.
      2. Acceptable evidence to establish the time of receipt at the Government installation includes the time/date stamp of that installation on the proposal wrapper, other documentary evidence of receipt maintained by the installation, or oral testimony or statements of Government personnel.
      3. If an emergency or unanticipated event interrupts normal Government processes so that proposals cannot be received at the office designated for receipt of proposals by the exact time specified in the solicitation, and urgent Government requirements preclude amendment of the solicitation, the time specified for receipt of proposals will be deemed to be extended to the same time of day specified in the solicitation on the first work day on which normal Government processes resume.
      4. Proposals may be withdrawn by written notice received at any time before award. Oral proposals in response to oral solicitations may be withdrawn orally. If the solicitation authorizes facsimile proposals, proposals may be withdrawn via facsimile received at any time before award, subject to the conditions specified in the provision at 52.215-5, "Facsimile Proposals." Proposals may be withdrawn in person by an offeror or an authorized representative, if the representative's identity is made known and the representative signs a receipt for the proposal before award.
        1. Unless otherwise specified in the solicitation, the offeror may propose to provide any item or combination of items.
        2. Offerors shall submit proposals submitted in response to this solicitation in English, unless otherwise permitted by the solicitation, and in U.S. dollars, unless the provision at FAR 52.225-17, Evaluation of Foreign Currency Offers, is included in the solicitation.
        3. Offerors may submit modifications to their proposals at any time before the solicitation closing date and time, and may submit modifications in response to an amendment, or to correct a mistake at any time before award.
        4. Offers may submit revised proposals only if requested or allowed by the Contracting Officer.
        5. Proposals may be withdrawn at any time before award. Withdrawals are effective upon receipt of notice by the Contracting Officer.
  4. Offer expiration date. Proposals in response to this solicitation will be valid for the number of days specified on the solicitation cover sheet (unless a different period is proposed by the offeror).
  5. Restriction on disclosure and use of data. Offerors that include in their proposals data that they do not want disclosed to the public for any purpose, or used by the Government except for evaluation purposes, shall —
    1. Mark the title page with the following legend:

      "This proposal includes data that shall not be disclosed outside the Government and shall not be duplicated, used, or disclosed–in whole or in part–for any purpose other than to evaluate this proposal." If, however, a contract is awarded to this offeror as a result of–or in connection with– the submission of this data, the Government shall have the right to duplicate, use, or disclose the data to the extent provided in the resulting contract. This restriction does not limit the Government's right to use information contained in this data if it is obtained from another source without restriction. The data subject to this restriction are contained in sheets [insert numbers or other identification of sheets]; and
    2. Mark each sheet of data it wishes to restrict with the following legend:

      "Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal."
  6. Contract award.
    1. The Government intends to award a contract or contracts resulting from this solicitation to the responsible offeror(s) whose proposal(s) represents the best value after evaluation in accordance with the factors and subfactors in the solicitation.
    2. The Government may reject any or all proposals if such action is in the Government's interest.
    3. The Government may waive informalities and minor irregularities in proposals received.
    4. The Government intends to evaluate proposals and award a contract after conducting discussions with offerors whose proposals have been determined to be within the competitive range. If the Contracting Officer determines that the number of proposals that would otherwise be in the competitive range exceeds the number at which an efficient competition can be conducted, the Contracting Officer may limit the number of proposals in the competitive range to the greatest number that will permit an efficient competition among the most highly rated proposals. Therefore, the offeror's initial proposal should contain the offeror's best terms from a price and technical standpoint.
    5. The Government reserves the right to make an award on any item for a quantity less than the quantity offered, at the unit cost or prices offered, unless the offeror specifies otherwise in the proposal.
    6. The Government reserves the right to make multiple awards if, after considering the additional administrative costs, it is in the Government's best interest to do so.
    7. Exchanges with offerors after receipt of a proposal do not constitute a rejection or counteroffer by the Government.
    8. The Government may determine that a proposal is unacceptable if the prices proposed are materially unbalanced between line items or subline items. Unbalanced pricing exists when, despite an acceptable total evaluated price, the price of one or more contract line items is significantly overstated or understated as indicated by the application of cost or price analysis techniques. A proposal may be rejected if the Contracting Officer determines that the lack of balance poses an unacceptable risk to the Government.
    9. If a cost realism analysis is performed, cost realism may be considered by the source selection authority in evaluating performance or schedule risk.
    10. A written award or acceptance of proposal mailed or otherwise furnished to the successful offeror within the time specified in the proposal shall result in a binding contract without further action by either party.
    11. If a post-award debriefing is given to requesting offerors, the Government shall disclose the following information, if applicable:
      1. The agency's evaluation of the significant weak or deficient factors in the debriefed offeror's offer.
      2. The overall evaluated cost or price and technical rating of the successful and the debriefed offeror and past performance information on the debriefed offeror.
      3. The overall ranking of all offerors, when any ranking was developed by the agency during source selection
      4. A summary of the rationale for award
      5. For acquisitions of commercial items, the make and model of the item to be delivered by the successful offeror.
      6. Reasonable responses to relevant questions posed by the debriefed offerors as to whether source-selection procedures set forth in the solicitation, applicable regulations, and other applicable authorities were followed by the agency.

(End of provision)

L.4 TYPE OF CONTRACT (APRIL 1984) (FAR 52.216-1)

The Government contemplates award of a cost reimbursement performance-based type contract resulting from this solicitation.

It is anticipated that one (1) contract award will be made from this solicitation and that the award is estimated to be made effective August 15, 2008.

L.5 SINGLE OR MULTIPLE AWARDS (OCT 1995)(FAR 52.216-27)

The Government may elect to award a single contract or to award multiple contracts for the same or similar supplies or services to two or more sources under this solicitation.

L.6 SERVICE OF PROTEST (AUG 1996)(FAR 52.233-2)

(a) Protests, as defined in Section 33.101 of the Federal Acquisition Regulation, that are filed directly with an agency, and copies of any protests that are filed with the General Accounting Office (GAO) shall be served on the Contracting Officer (addressed as follows) by obtaining written and dated acknowledgment of receipt from:

Director, Division of Contracts Management
Agency for Healthcare Research and Quality
540 Gaither Road
Rockville, Maryland 20850

(b)The copy of any protest shall be received in the office designated above within one day of filing a protest with the GAO.

L.7POINT OF CONTACT FOR TECHNICAL INQUIRIES

The technical contact for additional information and answering inquiries is the Contracting Officer. All questions regarding this solicitation shall be in writing and received by the Contracting Officer no later than 12:00 noon EST June 3, 2008. All questions should be e-mailed to Jessica Alderton at Jessica.Alderton@ahrq.hhs.gov. The subject line should be marked "Proposal Questions RFP No. AHRQ-08-10015."

L.8 PROPOSAL INTENT/ APPROVAL FOR CONTACT INFORMATION FOR BIDDERS LIST (Attachment 2)

It is requested that if an offeror intends to submit a proposal to this solicitation that the attached Proposal Intent Form be completed and returned to the address indicated by the date indicated. The submission of the intent form is not binding on an offeror to submit a proposal, nor does the failure to submit the form prohibit an offeror from submitting a proposal. The purpose is to provide us with an estimated number of proposals to assist us in our planning and logistics for proposal reviews.

We have added a request to include your contact information to a bidders list. The bidders list will be provided to interested offerors for subcontracting opportunities, as part of an amendment to the solicitation. In order for AHRQ to include your contact information on the bidders list, you must return the Proposal Intent Form and check the box that grants permission to add your name no later than the date listed above.

L.9 GENERAL INSTRUCTIONS

Introduction

The following instructions will establish the acceptable minimum requirements for the format and contents of proposals.

a. Contract Type and General Provisions: It is contemplated that a cost-reimbursement performance based contract will be awarded. In addition to the special provisions of this request for proposal (RFP), any resultant contract shall include the general clauses applicable to the selected offeror's organization and type of contract awarded. Any additional clauses required by Public Law, Executive Order, or procurement regulations, in effect at the time of execution of the proposed contract, will be included.

b. Authorized Official and Submission of Proposal: The proposal shall be signed by an official authorized to bind your (the offeror's) organization. Your proposal shall be submitted in the number of copies, to the address, and marked as indicated in the cover letter of this solicitation. Proposals will be typewritten, reproduced on letter sized paper and will be legible in all required copies.

c. Separation of Technical, Past Performance Information, and Business Proposal: The proposal shall be in 4 separate parts. To expedite the proposal evaluation, all documents required for responding to the RFP should be placed in the following order:

  1. TECHNICAL PROPOSAL: See Technical Proposal Instructions for recommended format (L.10). Please mark as original or copy.
  2. PAST PERFORMANCE INFORMATION: See Past Performance Information Instructions for format (L.11)
  3. BUSINESS PROPOSAL: See Business Proposal Instructions for recommended format (L.13).
  4. SMALL DISADVANTAGED BUSINESS PARTICIPATION PLAN See Small Disadvantaged Business Participation Plan Instructions for recommended format (L.12).

Each of the parts shall be separate and complete in itself so that evaluation of one may be accomplished independently of, and concurrently with, evaluation of the other.

d. Evaluation of Proposals: The Government will evaluate technical proposals in accordance with the criteria set forth in Section M, Evaluation/Award Criteria.

e. Rejection of Proposals: The Government reserves the right to reject any or all proposals received. It is understood that your proposal will become part of the official contract file.

f. Unnecessarily Elaborate Proposals: Unnecessarily elaborate brochures or other presentations beyond those sufficient to present a complete and effective proposal are not desired and may be construed as an indication of the offeror's lack of cost consciousness. Elaborate art work, expensive visual and other presentation aids are neither necessary nor wanted.

g. Privacy Act: The Privacy Act of 1974 (Public Law (P.L.) 93-579) requires that a Federal agency advise each individual whom it asks to supply information: 1) the authority which authorized the solicitation; 2) whether disclosure is voluntary or mandatory; (3) the principal purpose or purposes for which the information is intended to be used; (4) the uses outside the agency which may be made of the information; and 4) the effects on the individual, if any, of not providing all or any part of the requested information.

Therefore:

  1. The Government is requesting the information called for in this RFP pursuant to the authority provided by Section 301(g) of the Public Health Service Act, as amended, and P.L. 92-218, as amended.
  2. Provisions of the information requested are entirely voluntary.
  3. The collection of this information is for the purpose of conducting an accurate, fair, and adequate review prior to a discussion as to whether to award a contract.
  4. Failure to provide any or all of the requested information may result in a less than adequate review.
  5. The information provided by you may be routinely disclosed for the following purposes:
    • -to the cognizant audit agency and the Government Accountability Office (GAO) for auditing;
    • -to the Department of Justice as required for litigation;
    • -to respond to Congressional inquiries; and
    • -to qualified experts, not within the definition of Department employees for opinions as a part of the review process.

In addition, the Privacy Act of 1974 (P.L. 93-579, Section 7) requires that the following information be provided when individuals are requested to disclose their social security number.

Provision of the social security number is voluntary. Social security numbers are requested for the purpose of accurate and efficient identification, referral, review and management of AHRQ contracting programs. Authority for requesting this information is provided by Section 305 and Title IV of the Public Health Service Act, as amended.

h. The RFP does not commit the Government to pay any cost for the preparation and submission of a proposal. It is also brought to your attention that the Contracting Officer is the only individual who can legally commit the Government to the expenditure of public funds in connection with this or any acquisition action.

The Government reserves the right to award a contract without discussions if the Contracting Officer determines that the initial prices are fair and reasonable and that discussions are not necessary.

L.10 TECHNICAL PROPOSAL INSTRUCTIONS

The technical proposal shall contain an original and ten (10) copies. The technical proposal described below shall be limited to 100 pages not including biographic sketches, with no less than a 11 point font, double-spaced (lists of deliverables, person loading charts, and similar materials need not be double-spaced, so long as they are legible). Brief biographic sketches or CVs (less than ten pages in length) providing the relevant qualifications necessary for this effort are only required for key personnel. The technical proposal shall not contain reference to cost; however resources information, such as data concerning labor hours and categories, labor mix, materials, subcontracts, etc., shall be contained in the technical proposal so that your understanding of the Statement of Work (SOW) may be evaluated. It must disclose your technical approach in as much detail as possible, including, but not limited to, the requirements of these instructions. Lengthy proposals and voluminous appendices are neither needed nor desired as they are difficult to read and evaluate and may indicate the offeror's inability to concisely state their proposal. Appendices are to be provided electronically in MS Office format on CD, in the same quantity as the technical proposal.

a. Recommended Technical Proposal Format

The offeror's proposal should present sufficient information to reflect a thorough understanding of the work requirements and a detailed plan for achieving the objectives of the scope of work. Technical proposals shall not merely paraphrase the requirements of the Agency's scope of work or parts thereof, or use of phrases such as "will comply" or "standard techniques will be employed." The technical proposal must include a detailed description of the techniques and procedures to be used in achieving the proposed end results in compliance with the requirements of the Agency's scope of work.

  1. Cover Page: The name of the proposing organization, author(s) of the technical proposal, the RFP number and the title of the RFP should appear on the cover. The cover page must also include the DUNS and TIN as well as a point of contact and contact information. One (1) manually signed original copy of the proposal and the number of copies specified in the RFP cover letter are required.
  2. Table of Contents: Provide sufficient detail so that all important elements of the proposal can be located readily.
  3. Introduction: This should be a one or two page summary outlining the proposed work, your interest in submitting a proposal, and the importance of this effort in relation to your overall operation.
  4. Technical Discussion: The offeror shall prepare a technical discussion which addresses evaluation criteria A, B, C, D & E below (including their subcriteria). The offeror shall further state that no deviations or exceptions to the Statement of Work (SOW) are taken. The evaluation criteria (and their respective subcriteria) are as follows:

Technical proposals submitted in response to this RFP shall address each of the items described below, and shall be organized in the same manner and within the page limitations specified. Proposals shall be prepared in double-spaced format, with numbered pages.

A. Understanding the Problem

The Offeror shall provide a brief statement of the issues which underscore the concept of and need for this contract. Also included in this section shall be a description of the scope, purpose, products, and events called for under this contract. The offer shall include a discussion of the issues as they relate to Section 1013 of the Medicare Modernization Act of 2003. General discussion of technical approaches to the different types of activities identified in the RFP must be included.

B. Technical Approach

Briefly, but in sufficient detail to demonstrate Offeror's knowledge, the offeror shall describe its understanding of the objectives and underlying need for this contract, the role and responsibilities of a Clinical Decisions and Communications Science Center, and discuss approaches to the different types of activities. Offeror should clearly demonstrate experience in and ability to: (a) establish and implement a Decisions Sciences Center; (b) develop and pretest health information products and interactive decision aids that are derived from systematic reviews and other scientific information for various audiences; (c) conduct an audience assessment to inform product development, dissemination, and implementation strategies; (d) evaluate health information products and modes of dissemination (process and impact) and implementation; (e) plan and provide logistics for meetings (e.g., White Paper meetings and Symposiums); (f) develop a quarterly electronic newsletter; (g) disseminate and market products and events to increase awareness and use on a broad scale for users, particularly patients, providers, and policy makers; (h) develop a user's guide for decision makers; and (i) migrate and maintain an existing Web site.

The Offeror shall address the technical approach proposed for each task required by the Statement of Work.

C. Management Plan

Offeror shall demonstrate his/her ability to achieve the delivery of performance requirements through the proposed use of corporate management and other personnel resources as well as demonstrate that the Offeror's organizational structure and capabilities will meet the project's milestones in a timely manner. In doing so, and at a minimum, the offeror shall:

  1. Demonstrate corporate experience in managing projects of a similar size and nature.
  2. Provide a fully supported narrative showing offeror's understanding of the requirements in the Statement of Work from a managerial perspective. The narrative should at a minimum address the following topics:
    1. labor skill mix determination
    2. personnel selection and assignment;
    3. the percentage of full time core personnel (if a ratio of less than 70 percent full time core staff to 30 percent consultants/subcontractors is proposed, offeror shall provide a detailed explanation of how the proposed staffing plan ensures that the work is conducted by individuals with a mastery of the technical requirements of the Statement of Work.
    4. monitoring and control of services provided: technical quality, responsiveness, cost control, and effective and efficient resource utilization, compliance with technical requirement and contract provisions. Clearly show proposed system for quality control of work performed including documents to be produced, and proposed system for management control and contract provision compliance;
    5. managerial problems offeror expects to encounter. Describe the methods you propose to solve these problems. Demonstrate ability and flexibility to rapidly solve the same or similar managerial problems encountered previously;
    6. ability and flexibility to respond rapidly to changes in budget, priorities, and schedule.
  3. Indicate clear lines of authority and delineation of staff responsibilities.
  4. Describe the number of person hours for each task and for service delivery.
  5. Provide an organizational chart and a Program Evaluation Review Technique (PERT) chart showing all tasks (staffing plan).
  6. Describe coordination with proposed subcontractors, including monitoring of their performance.
  7. Provide a signed agreement, e.g., a letter of commitment, between the Offeror and any personnel other than current direct employees that includes dates of employment and specific tasks to be performed.
  8. Provide a person-level task-loading chart (to include consultants and subcontractors effort) and an organizational chart indicating clear lines of authority, delineating staff responsibilities and a plan for organizational backup. Employees not currently employed by the Offeror shall be listed with an asterisk (*).

D. Key Personnel

The proposal shall specify the project team, including subcontractors and consultants. In this project, the Project Director, and Project Manager (if used) are key personnel.

  1. Offeror shall provide evidence of the availability, qualifications, and demonstrated experience of key management personnel, including the Project Director, and Project Manager, if used. The Project Director should have, at a minimum, a doctoral or medical degree and have extensive experiences in developing health information tools and program management. The Project Director should not have less than ten (10) years total work experience which includes: 1) at least eight (8) years in the SOW's specialty services field in progressively supervisory positions; and 2) demonstrated skills in organizing and monitoring complex projects conducted by groups of diverse professionals.

    The Project Manager, if used, should have, at a minimum, a master's degree in a health and human services-related specialty and not less than ten (10) years total work experience which includes: 1) at least eight (8) years in the health education, health services research field, or behavior medicine; 2) demonstrated skills in developing information products for diverse audiences; 3) knowledge of the Medicaid, Medicare, and SCHIP programs; and 4) demonstrated skills in organizing and monitoring complex projects conducted by groups of diverse professionals.
    1. Describe how the education and technical experience of the Project Director, the Project Manager and other key technical personnel specifically relate to the SOW.
    2. Provide length and currency of the overall education of the Project Director, the Project Manager and other key technical personnel.
    3. Describe the experience of the proposed Project Director and the Project Manager in managing the SOW and complex projects involving scheduling multiple projects that require the participation of external partners, providing logistics support for off-site projects and small meetings, information system development and management, report development and production, and quality control. This description shall include at a minimum the size of projects managed, start-up time required, number of projects managed, problems encountered, and the resolution of those problems. Describe those projects currently managed. Describe how the management experience of the proposed Project Director and the Project Manager equips them to manage a staff which reflects the diversity of the SOW.
    4. Describe the ability of the proposed Project Director and the Project Manager to address issues of policy and legal sensitivity as they relate to the SOW.
  2. Offeror shall provide evidence of availability, qualifications, and demonstrated experience of key clinical, education, behavior, dissemination and technical personnel.
    1. Describe how the education and technical experience of the proposed technical personnel specifically relate to the SOW (e.g., writing for different audiences, developing and maintaining Web site, translating scientific information into information products including complex decision aids for a diverse audience, conducting literature reviews, managing meetings, planning and implementing events to include meetings, evaluating projects, quality assurance, audience assessments, marketing, dissemination, and use of products).
    2. Provide length and currency of the overall education of the proposed technical personnel.
    3. Describe the management experience of the technical personnel, if they are to serve as team leaders. Include a description of their experience in independent problem solving and conflict resolution, in facilitating groups in the analysis of large quantities of information, and in coordinating and editing the work of others in the production of extensive, complex reports. Describe those projects currently managed.
    4. Describe the ability of the technical personnel to address issues of health access to information, and health literacy and numeracy, as they relate to the SOW.

The Offeror is not required to have all types of expertise available on a full-time basis. To ensure top performance of the tasks set forth in the SOW, the Offeror must demonstrate that there will be personnel with experience in the following areas/disciplines either as core staff or as core staff plus rapid availability as consultants or subcontractors: general clinical health education, behavioral medicine, decision psychology, Web site programming and development, marketing and dissemination experience, communications science experience, basic epidemiology, health and medical information writing and editing, graphic design and production, qualitative and health services research, and meeting management.

E. Facilities

The Offeror shall describe the availability of facilities, space, and equipment necessary to adequately support the needs to successfully accomplish the projects goals and objectives. If the location where work will be performed is rented or leased, the Offeror shall so indicate and give the date that the lease ends.

L.11 PAST PERFORMANCE INFORMATION

Offerors shall submit the following information (original and 3 copies) as part of their proposal for both the offeror and proposed major subcontractors:

  1. A list of the last five (5) contracts and subcontracts completed (most relevant or most related) during the past three years and all contracts and subcontracts currently in process. Reference contracts and subcontracts completed during the past three years and include recently completed and ongoing work directly related to the requirements of this acquisition. Contracts listed may include those entered into by the Federal Government, agencies of State and local governments and commercial customers. Offerors that are newly formed entities without prior contracts should list contracts and subcontracts as required for all key personnel. Include the following information for each contract and subcontract:
    1. Name of contracting activity
    2. Contract number
    3. Contract type
    4. Total contract value
    5. Contract work
    6. Contracting Officer and telephone number
    7. Program Manager and telephone number
    8. Administrative Contracting Officer, if different from item f, and telephone number
    9. List of major subcontracts

  2. (2)The offeror must provide information on problems encountered on the contracts and subcontracts identified in (1) above and corrective actions taken to resolve those problems. Offerors should not provide general information on their performance on the identified contracts. General performance information will be obtained from the references.
  3. (3)The offeror may describe any quality awards or certifications that may indicate the offeror possesses a high-quality process for developing and producing the product or service required. Identify what segment of the company (one division or the entire company) that received the award or certification. Describe when the award or certification was bestowed. If the award or certification is over three years old, present evidence that the qualifications still apply.
  4. (4)Each offeror will be evaluated on his/her performance under existing and prior contracts for similar products or services. Performance information will be used for both responsibility determinations and as an evaluation factor against which offeror's relative rankings will be compared to assure best value to the Government. The Government will focus on information that demonstrates quality of performance relative to the size and complexity of the procurement under consideration. References other than those identified by the offeror may be contacted by the Government with the information received used in the evaluation of the offeror's past performance.

The attached Past Performance Questionnaire and Contractor Performance Form shall be completed by those organizations listed in (1) above. The evaluation forms shall be completed and forwarded directly to:

Jessica Alderton
Agency for Healthcare Research and Quality
Division of Contracts Management
540 Gaither Road
Rockville, Maryland 20850
FAX: 301-427-1740

Evaluation forms must be received by the date and time listed in order to be included in the review process. It is the responsibility of the offeror to ensure that these documents are forwarded to the Contracting Officer.

L.12SMALL DISADVANTAGED BUSINESS PARTICIPATION PLAN:

In accordance with FAR Part 15.304(c)4, the extent of participation of Small Disadvantaged Business (SDB) concerns in performance of the contract shall be evaluated in unrestricted acquisitions expected to exceed a total estimated cost of $550,000 ($1,000,000 for construction) subject to certain limitations (see FAR 19.201 and 19.1202).

A. All offerors, regardless of size, shall submit the following information in original and 2 copies.

A plan on the extent of participation of Small Disadvantaged Business concerns in performance of the contract. Participation in performance of the contract includes the work expected to be performed by SDB concern(s). This can include SDB (as prime contractor), joint ventures, teaming arrangements, and subcontracts. Include the following information in SDB participation plans:

  1. The extent of an offeror's commitment to use SDB concerns. Commitment should be as specific as possible, i.e., are subcontract arrangements already in place, letters of commitment, etc. Enforceable commitments will be weighted more heavily than non-enforceable ones.
  2. Specifically identify the SDB concerns with point of contact and phone number.
  3. The complexity and variety of the work SDB concerns are to perform.
  4. Realism for the use of SDB in the proposal.
  5. Past performance of the Offeror in complying with subcontracting plans for SDB concerns.
  6. Targets expressed as dollars and percentage of total contract value for each participating SDB; which will be incorporated into and become part of any resulting contract.
  7. The extent of participation of SDB concerns in terms of the total acquisition.

B. SDB participation information will be used for both responsibility determinations and as an evaluation factor against which offeror's relative rankings will be compared to assure the best value to the Government. The Government will focus on information that demonstrates realistic commitments to use SDB concerns relative to the size and complexity of the acquisition under consideration. The Government is not required to contact all references provided by the offeror. Also, references other than those identified by the offeror may be contacted by the Government to obtain additional information that will be used in the evaluation of the offeror's commitment to SDB participation.

L.13 BUSINESS PROPOSAL

The offeror shall submit as part of the proposal a separate enclosure titled "Business Proposal." The Business Proposal shall include the Cost/Price Proposal, the Small Business Subcontracting Plan, and Other Administrative Data in accordance with the following:

A.Cost/Price Proposal

A cost proposal shall be submitted in accordance with FAR 15, in a format similar to the attachment. The offeror's own format may be utilized, but all required information in the attachment shall be provided.

The business proposal must contain sufficient information to allow the Government to perform a basic analysis of the proposed cost or price of the work. This information shall include the amounts of the basic elements of the proposed cost or price.

As appropriate, cost breakdowns shall be provided for the following cost elements.

  1. Direct Labor
    The estimated cost for all personnel who will be assigned for direct work on this project shall be included. Give the name, title, percent of effort or time, salary and fringe benefits for each employee.
    Salary increases that are anticipated during performance of a resultant contract should be proposed as a cost. If escalation is included, state the degree (percent) and methodology, e.g., annual flat rate applied to a base rate as of a specific date or a mid-pointed rate for the period of performance. State whether any additional direct labor (new hires) will be required during the performance period of this procurement. If so, state the number required and anticipated date of hire. Also, specify the month and day on which your fiscal year commences.
  2. Supplies and Equipment
    Include description, unit price, quantity, total price, justification for purchasing or leasing items and the basis for pricing (vendor quotes, invoices prices, etc.).
  3. Travel
    The amount proposed for travel shall be supported with a breakdown which includes purposes, destination, duration, and estimated cost (transportation and per diem) for each proposed trip. If travel costs are proposed on the basis of your organization's established travel policy, a copy of the policy must be provided.
  4. Consultants
    This element should include name(s) of consultant, number of days, and daily rate. The method of obtaining each consultant, either sole source or competitive, and the degree of competition or the rationale for sole source shall be explained.
  5. Subcontractors
    Subcontractor costs shall be broken down and supported by cost and pricing data adequate to establish the reasonableness of the proposed amount. Subcontract cost detail should be similar to the level of detail provided for the prime contractor, with the same cost elements. Support documentation should include degree of subcontract competition and basis for selecting source.
  6. Other Direct Costs
    Any proposed other direct costs shall be supported with breakdown outlining the separate costs proposed and details supporting the formulation of the costs proposed. A signed agreement between the offeror and any personnel other than direct employees that includes dates of employment, salary, and specific tasks to be performed should be included.
  7. Indirect Costs
    Indicate how you have computed and applied indirect costs, and provide a basis for evaluating the reasonableness of the proposed rates.

B. Small Business Subcontracting Plan:

All offerors except small businesses are required to submit a subcontracting plan in accordance with the Small Business Subcontracting Plan, FAR 52.219-9, incorporated in this solicitation. If the model plan is not used, all elements outlined must be addressed in the offeror's format. If the offeror is not a small business and fails to submit a subcontracting plan with the initial proposal, the offeror will be considered nonresponsive and their proposal will be returned without further consideration.

This provision does not apply to small business concerns. This provision does apply to all other offerors, including large business concerns, colleges, universities and non-profit organizations.

The term "subcontract" means any agreement (other than one involving an employer-employee relationship) entered into by a Federal Government prime contractor or subcontractor calling for supplies or services required for the performance of the original contract or subcontract. This includes, but is not limited to, agreements/ purchase orders for supplies and services such as equipment purchase, copying services, and travel services.

The offeror understands that:

  1. No contract will be awarded unless and until an acceptable plan is negotiated with the Contracting Officer. The plan will be incorporated in to the contract.
  2. An acceptable plan must, in the determination of the Contracting officer, provide the maximum practicable opportunity for small business concerns and small business concerns owned and controlled by socially and economically disadvantaged persons to participate in the performance of the contract.
  3. If a subcontracting plan acceptable to the Contracting Officer is not negotiated within the time limits prescribed by the contracting activity and such failure arises out of causes within the control and with the fault or negligence of the offeror, the offeror shall be ineligible for award. The Contracting Officer shall notify the Contractor in writing of the reasons for determining a subcontracting plan unacceptable early enough in the negotiation process to allow the Contractor to modify the plan within the time limits prescribed.
  4. Prior compliance of the offeror with other such subcontracting plans under previous contracts will be considered by the Contracting Officer in determining the responsibility of the offeror for award of the contract.
  5. It is the offeror's responsibility to develop a satisfactory subcontracting plan with respect to small business concerns and small business concerns owned and controlled by socially and economically disadvantaged individuals, and women-owned small business concerns, and that each such aspect of the offeror's plan will be judged independent of the other.
  6. The offeror will submit, as required by the Contracting Officer, subcontracting reports in accordance with the instructions thereon, and as further directed by the Contracting Officer. Subcontractors will also submit these reports to the Government Contracting Officer or as otherwise directed, with a copy to the prime Contractor's designated small and disadvantaged business liaison.
  7. For this particular acquisition, the AHRQ recommended goal (as a percentage of total contract value for the base period) is 20% for Small Businesses, which shall included at least 5.5% (as a percentage of total planned subcontract dollars for the base period) for Small Disadvantaged Businesses, at least 5% (as a percentage of total planned subcontract dollars total planned subcontract dollars for the base period) for Women-Owned Small Businesses, and at least 3% (as a percentage of total planned subcontract dollars for the base period) for HUBZone Small Businesses and at least 3% (as a percentage of total planned subcontract dollars for the base period) for Veteran-Owned Small Businesses. These goals represent AHRQ's expectations of the minimum level for subcontracting with small business at the prime contract level. Any goal stated less than the AHRQ recommended goal shall be justified and is subject to negotiation.

C. Other Administrative Data

  1. Terms and Conditions: The proposal shall stipulate that it is predicated upon the terms and conditions of the RFP. In addition, it shall contain a statement to the effect that it is firm for a period of at least 120 days from the date of receipt thereof by the Government.

    Minimum Bid Acceptance Period (April 1984)
    1. "Acceptance period," as used in this provision, means the number of calendar days available to the Government for awarding a contract from the date specified in this solicitation for receipt of bids.
    2. This provision supersedes any language pertaining to the acceptance period that may appear elsewhere in this solicitation.
    3. The Government requires a minimum acceptance period of 120 days.
    4. A bid allowing less than the Government's minimum acceptance period may be rejected.
    5. The bidder agrees to execute all that it has undertaken to do, in compliance with its bid, if that bid is accepted in writing within (i) the acceptance period stated in paragraph (3) above, or (ii) any longer acceptance period stated in paragraph (4) above.
  2. Authority to Conduct Negotiations: The proposal shall list the names and telephone numbers of persons authorized to conduct negotiations and to execute contracts.
  3. Property:
    1. It is HHS policy that contractors will provide all equipment and facilities necessary for performance of contracts. Exception may be granted to furnish Government-owned property, or to authorize purchase with contract funds, only when approved by the contracting officer. If additional equipment must be acquired, you shall include the description, estimated cost of each item and whether you will furnish such items with your own funds.
    2. You shall identify Government-owned property in your possession and/or property acquired from Federal funds to which you have title, that is proposed to be used in the performance of the prospective contract.
    3. The management and control of any Government property shall be in accordance with HHS Publication (OS) 74-115 entitled, Contractor's Guide for Control of Government Property" 1990, a copy of which will be provided upon request.
  4. Royalties: You shall furnish information concerning royalties which are anticipated to be paid in connection with the performance of work under the proposed contract.
  5. Commitments: You shall list other commitments with the Government relating to the specified work or services and indicate whether these commitments will or will not interfere with the completion of work and/or services contemplated under this proposal.
  6. Financial Capacity: You shall provide sufficient data to indicate that you have the necessary financial capacity, working capital, and other resources to perform the contract without assistance from any outside source. If not, indicate the amount required and the anticipated source. (Financial data such as balance sheets, profit and loss statements, cash forecasts, and financial histories of your organization's affiliated concerns should be utilized.)
  7. Performance Capability: You shall provide acceptable evidence of your "ability to obtain" equipment, facilities, and personnel necessary to perform the requirements of this project. If these are not represented in your current operations, they should normally be supported by commitment or explicit arrangement, which is in existence at the time the contract is to be awarded, for the rental, purchase, or other acquisition of such resources, equipment, facilities, or personnel. In addition, you shall indicate your ability to comply with the required or proposed delivery or performance schedule taking into consideration all existing business commitments, commercial as well as Government.
  8. Representations and Certifications: Section K, "Representations and Certifications and Other Statements of Offerors" shall be completed and signed by an official authorized to bind your organization. Section K shall be made a part of the original business proposal.

L.14 SELECTION OF OFFERORS

  1. The acceptability of the technical portion of each contract proposal will be evaluated by a technical review committee. The committee will evaluate each proposal in strict conformity with the evaluation criteria of the RFP, utilizing point scores and written critiques. The committee may suggest that the Contracting Officer request clarifying information from an offeror.
  2. The business portion of each contract proposal will be subjected to a limited cost review, management analysis, etc.
  3. Past performance of the technically acceptable offerors will be evaluated by AHRQ staff. A competitive range will be determined. The competitive range will consist of those offers which are highly rated, based upon the technical and past performance evaluation. Oral or written discussions will be conducted with all offerors in the competitive range, if necessary. All aspects of the proposals are subject to discussions, including cost, technical approach, past performance, and contractual terms and conditions. Final Proposal Revisions may be requested with the reservation of the right to conduct limited negotiations after submission of Final Proposal Revisions.

    A cost realism analysis to determine the cost of performance of each offeror cannot be made for the base contract award. The base contract award will be awarded with no dollars, individual task orders will reflect the value of the specific work to be performed. Each task order will be negotiated on the basis of the work to be performed and the costs proposed. The proposed costs will include specific personnel and the corresponding rate within the Labor Category Hourly Rate Range. The proposed costs for a particular task order may also include specific other direct costs, as necessary for performance of the task (i.e. travel, consultants, and consumables). These will be reviewed and negotiated on an as needed basis.

  4. A final best-value analysis will be performed taking into consideration the results of the technical evaluation, cost analysis, past performance, and ability to complete the work within the Government's required schedule. The Government reserves the right to make an award to the best advantage of the Government, technical merit, cost, past performance, and other factors considered.
  5. The Government reserves the right to make a single award, multiple awards, or no award at all to the RFP.

Return to Contents

SECTION M - EVALUATION FACTORS FOR AWARD

TECHNICAL EVALUATION CRITERIA

Selection of an offeror for contract award will be based on an evaluation of proposals against the evaluation criteria and award will be made to that responsible offeror whose proposal is most advantageous to the Government. Evaluation Criteria 1 through 4 will be evaluated by a peer review technical committee that will also recommend technical acceptability or unacceptability of the proposal. Offerors that submit technically acceptable proposals will then be evaluated for past performance and Small Disadvantaged Business Participation Plan. Following these evaluations a competitive range will be determined.

All evaluation factors, other than cost or price, when combined are significantly more important than cost or price. However, cost/price may become a critical factor in source selection in the event that two or more offerors are determined to be essentially equal following the evaluation of all factors other than cost or price. While the scientific technical merit of the proposals will receive paramount consideration in the selection of the Contractor for this acquisition, the Government may also consider other factors in source selection.

In any event, the Government reserves the right to make an award to that offeror whose proposal provides the best overall value to the Government. The Government reserves the right to make a single award, multiple awards, or no award at all.

THE GOVERNMENT RESERVES THE RIGHT TO MAKE AN AWARD WITHOUT DISCUSSION

The Government reserves the right to make an award to the best advantage of the Government.
The evaluation factors and assigned weights which will be used in the overall review of the offeror's proposal are outlined below. The technical proposal shall consist of the responses to evaluation criteria 1 through 5 (including subcriteria). The offeror should show that the objectives stated in the proposal are understood and offer a logical program for their achievement. The following criteria will be used to evaluate proposals and will be weighted as indicated in establishing a numerical rating for all proposals submitted. Factors facilitating the evaluation of each criterion below are referenced in the corresponding criteria found in Section L of this solicitation.

OFFERORS PLEASE NOTE: Evaluation Criteria 1 through 5, for a total of 100 points, will be evaluated by a peer review technical committee that will also recommend technical acceptability or unacceptability of the proposal. Program staff and contracting personnel will review and evaluate Criteria 6 and 7 for a total of 50 points. The total possible points for Evaluation Criteria 1 through 7 is 150 points.


EVALUATION CRITERIA                                                           WEIGHT

A.  Understanding the Problem                                                    5

The proposal shall be evaluated on the completeness of the proposal and the offerors demonstrated understanding of the problems of the project in its response to the objectives and tasks and solution approach thereto, specifically as it relates to Section 1013 of the Medicare Modernization Act.  The offeror will be evaluated on its discussion of the technical approaches to the different types of activities identified.

B.  Technical Approach                                                                35

The proposal shall be evaluated on the completeness, reasonableness, clarity, and feasibility of the approach to satisfy the requirement of each individual task in the Statement of Work.  Specific attention will be paid to the extent to which the offeror clearly demonstrates its experience, creativity, and ability to:

  • Establish and implement a Decisions Sciences Center
  • Extent to which the proposal demonstrates a capability to provide continuity of support over the multi-year period of performance required of the Decisions Sciences Center
  • Approach to working with, and providing technical assistance to a diverse audience, particularly patients, providers, and policymakers
  • Developing and pretesting information products, decision aids and other tools for a diverse audience
  • Plan meetings and other events
  • Developing and maintaining a Web site
  • Evaluating (process and impact) health information products and modes of dissemination
  • Conducting an audience assessment
  • Dissemination, marketing, and implementation of products, services and events

C. Management Plan                                                                  25

The Offeror's demonstrated ability to achieve the delivery of performance requirements through the proposed use of corporate/organizational management and other personnel resources will be evaluated.  The offeror's demonstrated ability to manage subcontractors and consultants, and the ability to complete the project milestones using a cost-effective approach will be evaluated.

D. Key Personnel                                                                        30

The proposal will be evaluated on the demonstrated background, skills, professional experience and education of key personnel, consultants, and subcontractors in the area of clinical health education, behavioral medicine, decision psychology, marketing and web site services and development.  Background, skills and experience shall be evaluated in: developing information tools for a diverse audience; developing decision aids; marketing and dissemination, providing technical assistance; conducting process and impact evaluations; developing and maintaining Web sites; interacting with AHRQ programs. 

Offerors will also be evaluated on:

  • Availability of scientific personnel with the breadth and depth of clinical specialties and health systems; working with patients, providers and policy makers.
  • Overall degree to which Offeror is able to provide the range of professional, technical, management and other personnel, both in leadership positions and support positions, with required experience and expertise to meet the requirements for work envisioned under this contract.
  • Percentage and category(ies) of personnel who are (a) full-time or part-time employees, (b) consultants, or (c) subcontractors.

E. Facilities                                                                                5

Proposals will be evaluated on the availability of adequate facilities, space, and equipment (computers, word-processing, photocopying, facsimile) for accomplishing the project goals and objectives.  In addition to computer hardware, the contractor must provide necessary computer software capability.

TOTAL POINTS BEFORE PAST PERFORMANCE                                100

 

5. Past Performance                                                                               45

Offerors will be evaluated on all past performance (since January 1, 2003). Completed questionnaires will provide a basis for determining past performance evaluation as well as information obtained from the references listed in the proposal, other customers known to the Government, consumer protection organizations, and any others who may have useful and relevant information. Information will also be considered regarding any significant subcontractors and key personnel records.

The offerors past performance will be evaluated on the basis of the following factors:

  1. Quality
    How well has the offeror conformed to the performance standard in providing the services or achieving the stated objective(s) of contracts or grants? Quality will be evaluated by the personnel provided, the level of effort agreed to in the contract statement of work or grant, quality of final products (e.g., findings, tools), implementation activities, dissemination and activities that promote turning research into action (i.e. promoting uptake of innovation).

  2. Timeliness
    How well has the offeror adhered to timetables and delivery schedules in providing the required services or products? Consideration is given to the offeror's efforts to recommend and/or take corrective actions to keep work on schedule.

  3. Business Relations/ Customer satisfaction
    The offeror will be rated on professional and cooperative behavior with the client.

  4. Cost control
    The offeror will be rated on the ability to set reasonable budgets within contracting or grant guidelines and adhere to them in conducting research.

Assessment of the offeror's past performance will be one means of evaluating the credibility of the offeror's proposal, and relative capability to meet performance requirements.

Evaluation of past performance will often be quite subjective based on consideration of all relevant facts and circumstances. It will not be based on absolute standards of acceptable performance. The Government is seeking to determine whether the offeror has consistently demonstrated a commitment to customer satisfaction and timely delivery of services and quality products at fair and reasonable prices.

The assessment of the offeror's past performance will be used as a means of evaluating the relative capability of the offeror and the other competitors. Thus, an offeror with an exceptional record of past performance may receive a more favorable evaluation than another whose record is acceptable, even though both may have acceptable technical proposals.

By past performance, the Government means the offeror's record of conforming to specifications and to standards of good workmanship; the offeror's record of forecasting and controlling costs; the offeror's adherence to contract schedules, including the administrative aspects of performance; the offeror's reputation for reasonable and cooperative behavior and commitment to customer satisfaction; and generally, the offeror's business-like concern for the interest of the customer.

The Government will consider the number or severity of an offeror's problems, the effectiveness of corrective actions taken, the offeror's overall work record, and the age and relevance of past performance information.

If the offeror or the proposed employees for the offeror, do not have a past performance history relative to this acquisition, or past performance not relative to this acquisition, the offeror will not be evaluated favorably or unfavorably on this factor. A neutral rating will be determined.

The Government reserves the right to evaluate relevant past performance information not specifically provided by the offeror.

In evaluating past performance the Government, will consider the offeror's effectiveness in quality of products or services; timeliness of performance; cost control; business practices; customer satisfaction, and key personnel past performance.

NOTICE: Past Performance questionnaires are to be provided to the Contracts Office NO LATER than the closing date and time for receipt of proposals. It is the offeror's responsibility to ensure that these documents are forwarded to Jessica Alderton by email: jessica.alderton@ahrq.hhs.gov.

6.Small Disadvantaged Business Participation Plan                              5

The evaluation will be abased on information obtained from the plan provided by the offeror, the realism of the proposal and other relevant information obtained from the named SDB concerns, and any other information supplies by the offeror concerning problems encountered in SDB participation.

Evaluation of the SDB Participation Plan will be a subjective assessment based on consideration of all relevant facts and circumstances. It will not be based on absolute standards of acceptable performance. The government is seeking to determine whether the offeror has demonstrated a commitment to use SDB concerns for the work that it intends to perform as the prime contractor.

The assessment of the offeror's SDB Participating Plan will be used as a means of evaluating the relative capacity and commitment of the offeror and the other competitors. Thus, an offeror with an exceptional record of participation with SDB concerns may receive more points and a more favorable evaluation than another whose record is acceptable , even though both may have acceptable technical proposals SDB participation will be scored with offerors receiving points from 0 to 5, with 5 being the most favorable.

TOTAL AVAILABLE POINTS............................................................................. 150


Return to Contents

ATTACHMENT 1

PAST PERFORMANCE QUESTIONNAIRE

PART ONE: INSTRUCTIONS

The offeror listed below has submitted a proposal in response to the Agency for Healthcare Research and Quality (AHRQ) Solicitation No. AHRQ-08-10015, entitled "The John M. Eisenberg Center for Clinical Decisions and Communications Sciences." Past performance is a very important part of the evaluation criteria for this acquisition, so input from previous customers of the offeror is extremely important. This office would greatly appreciate you taking the time to complete this form. This information is to be provided to Jessica Alderton, the AHRQ Contracting Officer and is NOT to be disclosed to the offeror either verbally or in writing. Please provide an honest assessment and return to AHRQ (either by mail, fax or email), no later than July 9, 2008, 12 noon EST. If you have any questions, please contact Jessica Alderton at via e-mail Jessica.Alderton@ahrq.hhs.gov.

Jessica Alderton
Agency for Healthcare Research and Quality
Division of Contracts Management
540 Gaither Road
Rockville, Maryland 20850
FAX: (301) 427-1740

NAME OF OFFEROR:_____________________________________


ADDRESS:_____________________________________________
______________________________________________________


                                                                     

 Contractor Performance Form

1.        Name of Contractor:_____________________________________

2.        Address:______________________________________________
          ______________________________________________________
           ______________________________________________________

3.        Contract/Grant Number: __________________________________

4.        Contract/Grant Value (Base Plus Options): ________________________

5.        Contract/Grant Award Date: ____________________________________

6.        Contract/Grant Completion Date: ________________________________

7.        Type of Contract/Grant: (Check all that apply) (  )FP (  )FPI (  )FP-EPA
            (  ) Award Fee (  ) CPFF-Completion (  ) CPFF-Term (  ) CPIF (  ) CPAF
            (  ) IO/IQ (  ) BOA (  ) Requirements (  ) Labor-Hour (  )T&M (  ) SBSA
            (  )8(a) (  )SBIR (  ) Sealed Bid(  )Negotiated(  )Competitive (  )Non-Competitive
           
8.        Description of Requirement:


CONTRACTOR'S PERFORMANCE RATING

Ratings: Summarize contractor performance and circle in the column on the right the number which corresponds to the performance rating for each rating category.  Please see reverse page for explanation of rating scale.

 

Category Comments Rating Scale

Quality of Product or Service

Comments

0
1
2
3
4
5

Cost
Control

Comments 0
1
2
3
4
5

Timeliness of
Performance

Comments 0
1
2
3
4
5
Business Relations Comments 0
1
2
3
4
5
     

Customer Satisfaction - Is/was the Contractor committed to customer satisfaction? ___Yes  ___No ;

Would you use this Contractor again? ___Yes   ___No

Reason:     


NAME OF EVALUATOR:______________________________________________
            (Please Print)______________________________________________

 

TITLE OF EVALUATOR:______________________________________________

 

SIGNATURE OF EVALUATOR:_________________________________________

DATE:______________________________________________

MAILING ADDRESS:  Include name of organization/ federal agency

______________________________________________

______________________________________________

______________________________________________

PHONE #:______________________________________________

E-MAIL:______________________________________________

 


Rating Guidelines:  Summarize contractor performance in each of the rating areas. Assign each area a rating 0(Unsatisfactory), 1(Poor), 2(Fair),  3(Good),  4(Excellent)  5(Outstanding).  Use the following instructions as guidance in making these evaluations.

  Quality Cost Control

Timeliness of Performance

Business Relation

  -Compliance with contract requirements
-Accuracy of reports
-Technical excellence
-Within budget(over/under target costs)
-Current, accurate, and complete billings
-Relationship of negotiated costs to actual
-Cost efficiencies
-Change orders issue
-Met interim milestones
-Reliable
-Responsive to technical direction
-Completed on time, including wrap-up and contract adm
-No liquidated damages assessed
-Effective management
-Businesslike correspondence
-Responsive to contract requirements
-Prompt notification of problems
-Reasonable/cooperative
-Flexible
-Pro-active
-Effective small/small disadvantaged business sub-contracting program
0-unsatisfactory Nonconformances are jeopardizing the achievement of contract requirements, despite use of Agency resources Ability to manage cost issues is jeopardizing performance of contract requirements, despite use of Agency resources Delays are jeopardizing the achievement of contract requirements, despite use of Agency's resources Response to inquiries, technical/service/administrative issues is not effective
1-Poor Overall compliance requires major Agency
resources to ensure achievement of contract requirements
Ability to manage cost issues requires major Agency resources to ensure achievement of contract requirements Delays require major Agency resources to ensure achievement of contract requirements Response to inquiries, technical/service/administrative issues is marginally effective
2-Fair Overall compliance requires minor Agency resources to ensure achievement of contract requirements Ability to manage cost issues requires minor Agency resources to ensure achievement of contract requirements Delays require minor Agency resources to ensure achievement of contract requirements Response to inquiries, technical/service/administrative issues is somewhat effective
3-Good Overall compliance does not impact achievement of contract requirements Management of cost issues does not impact achievement of contract requirements Delays do not impact achievement of contract requirements Response to inquiries, technical/service/administrative issues is usually effective
4-Excellent There are no quality problems There are no cost management issues There are no delays Response to inquiries, technical/service/administrative issues is effective

5-Outstanding. The Contractor has demonstrated an outstanding performance level that justifies adding a point to the score. It is expected that this rating will be used in those rare circumstances where Contractor performance clearly exceeds the performance levels described as "Excellent."


Return to Contents

ATTACHMENT 2

PROPOSAL INTENT RESPONSE SHEET

RFP No. AHRQ-08-10015
                                                                                               
Please review the attached request for proposal.  Furnish the information requested below and return this page by June 10, 2008.  Your expression of intent is not binding but will greatly assist us in planning for the proposal evaluation.

==================================================================

[  ]  INTEND TO SUBMIT A PROPOSAL

[  ] DO NOT INTEND TO SUBMIT A PROPOSAL FOR THE FOLLOWING REASONS:

 

 

==================================================================

[  ] I GRANT PERMISSION TO THE AGENCY FOR HEALTHCARE RESEARCH AND QUALITY, CONTRACTS OFFICE TO ADD THE CONTACT INFORMAION BELOW TO A BIDDERS LIST TO PROVIDE TO OTHER INTERESTED OFFERORS FOR TEAMING/SUBCONTRACING OPPORTUNITIES. (*MUST INCLUDE AUTHORIZED SIGNATURE)

COMPANY/INSTITUTION NAME & ADDRESS:

 

 

*AUTHORIZED SIGNATURE: __________________________________

TYPED/PRINT NAME AND TITLE: ______________________________

DATE: ___________________

==================================================================

[  ] PLEASE DO NOT RELEASE THE CONTACT INFORMATION.

Please return to: 

Jessica Alderton
Agency for Healthcare Research and Quality
Contracts Management
540 Gaither Road
Rockville, Maryland 20850


Return to Contents

Attachment 3

BREAKDOWN OF PROPOSED ESTIMATED COST (PLUS FEE) AND LABOR HOURS 

INSTRUCTIONS FOR USE OF THE FORMAT

  1. Refer to Business Proposal Instructions, Section L of this solicitation. The Instructions contain the requirements for proper submission of cost/price data which must be adhered to.
  2. This sample format has been prepared as a universal guideline for all solicitations. It may require amending to meet the specific requirements of this solicitation.  For example, this solicitation may require the submission of cost/price data for three years listed on this form.  (See Section L, Instructions, Conditions and Notices to Offerors, for the estimated duration of this project.)  If this solicitation is phased, identify each phase in addition to each year.  Total each year, phase, and sub-element.
  3. This format must be used to submit the breakdown of all proposed estimated cost elements. List each cost element and sub-element for direct costs, indirect costs and fee, if applicable. In addition, provide detailed calculations for all items. For example:
    1. For all personnel, list the name, title, rate per hour and number of hours proposed. If a pool of personnel is proposed, list the composition of the pool and how the cost proposed was calculated. List the factor used for prorating Year One and the escalation rate applied between years.

      Offeror's proposal should be stated in the same terms as will be used to account for and record direct labor under a contract (i.e. percentage of effort is used for most faculty and professional employees at educational institutions).  If percentages of effort are used, the basis to which such percentages are applied must also be submitted by the offeror.  The attached format should be revised to accommodate direct labor proposed as a percentage of effort.

    2. For all materials, supplies, and other direct costs, list all unit prices, etc., to detail how the calculations were made.
    3. For all indirect costs, list the rates applied and the base the rate is applied to.
    4. For all travel, list the specifics for each trip.
    5. For any subcontract proposed, submit a separate breakdown format.
    6. Justification for the need of some cost elements may be listed as an attachment, i.e., special equipment, above average consultant fees, etc.
  4. If the Government has provided "uniform pricing assumptions" for this solicitation, the offeror must comply with and identify each item.

RFP Number:                                              

Organization:                                              

Date:    

BREAKDOWN OF PROPOSED ESTIMATED COST (PLUS FEE) AND LABOR HOURS

COST ELEMENT   Option Option Total
Year 1 Year 2 Year 3 Year 1 Year 2 Year 3

DIRECT LABOR:

Labor Category   
(Title and Name—use additional pages as necessary)

Rate Hours
Amt
Hours
Amt
Hours
Amt
Hours
Amt
Hours
Amt
Hours
Amt
 
DIRECT LABOR COST:   $___ $___ $___ $___ $___ $___  
MATERIAL COST:   $___ $___ $___ $___ $___ $___  
TRAVEL COST:   $___ $___ $___ $___ $___ $___  
OTHER (Specify)   $___ $___ $___ $___ $___ $___  
OTHER (Specify)   $___ $___ $___ $___ $___ $___  
TOTAL DIRECT COST:   $___ $___ $___ $___ $___ $___  

FRINGE BENEFIT COST:
(
if applicable)

               
    % of Direct Labor Cost   $___ $___ $___ $___ $___ $___  
INDIRECT COST:                
    % of Total Direct Cost   $___ $___ $___ $___ $___ $___  
TOTAL COST:   $___ $___ $___ $___ $___ $___  

FIXED & AWARD FEES:
(if applicable)

               
    % of Total Est. Cost   $___ $___ $___ $___ $___ $___  
GRAND TOTAL EST COST   $___ $___ $___ $___ $___ $___  

Return to Contents

Attachment 4. Performance Requirements Summary

 

Performance Factor Key Performance Standards Surveillance Methods
1. Project Management. a. Project work plan is clear, complete and useful; and contains the following: (1) detailed approach to performing each of the tasks outlined in the SOW; (2) projected timeline for performing each task and subtask; (3) plan for transition from the prior contractor; (4) proposed organization and staffing; and (5) proposed mechanisms for frequent communications with the Scientific Resource Center, EHC contractors and grantees, OCKT, and the PO. Work plan is responsive to PO guidance provided in orientation and transition meetings.
b. Project work plan is submitted on timely basis, in accordance with Schedule of Deliverables.
c. Updates to project work plan are clear, complete and useful; and reflective of changes in project requirements, actual experience, and PO guidance.
d. Contractor provides useful call agenda for weekly conference calls with PO within 1 working day of call, and summary of call highlights and action items by 2 working days after call.
e. Monthly progress reports that are clear, complete, concise and useful and contain: (1) activities performed within each task; (2) barriers or problems in performance of tasks and how they were handled; (3) labor hours expended and estimated costs; and (4) any issues of which AHRQ should be aware. Monthly progress reports are submitted on timely basis, in accordance with Schedule of Deliverables.
f. Quarterly reports that are clear, complete, concise and useful and contain: (1) description of project accomplishments for the previous quarter; (2) activities planned for the next quarter; and (3) proposed changes in delivery schedule, as appropriate. Quarterly reports are submitted on timely basis, in accordance with Schedule of Deliverables.
g. Contractor notifies PO promptly of any problems (technical, schedule, staffing, or cost) that could impact successful completion of the individual tasks or deliverables. Contractor clearly indicates the potential impact of the problem(s) on the schedule, cost, or content of the required services or deliverables; and recommends practical solutions.
h. Contractor provides clear, timely and useful responses to PO questions on project schedule, budget, technical approaches and other issues.
i. Contractor provides project staff that demonstrate the knowledge, skills and experience appropriate for their respective tasks.
j. Contractor provides a sufficient number of project staff for the various tasks.
k. Contractor effectively manages subcontractor(s), if applicable, in ensuring timely delivery of subcontract products/services, and resolution of any potential performance or capacity problems.
l. Contractor provides clear, accurate, complete and timely invoices, as described in the contract.
m. Contractor establishes effective working relationships with the Scientific Resource Center, EPCs, and EHC contractors and grantees to ensure awareness of different topics and centers working on assigned topic.
n. Contractor participates and provides effective contributions (including presenting current work) in meetings of EPCs, DEcIDE Research Centers, EHC stakeholders, and monthly EHC calls.

Review of deliverables by AHRQ Project Officer (PO) and other Government staff.

Review of Contractor's responsiveness and timeliness in following up to AHRQ PO's inquiries and requests.

2. Information Needs Assessment and Product Evaluation. a. Contractor proposes innovative approaches in conducting and implementing assessment of information needs.
b. Contractor conducts assessment of information needs of target audiences (patients, providers, and policymakers) that is clear, complete and useful; and effectively addresses issues such as literacy, numeracy, culture and ethnicity. Assessment addresses impediments to understanding and using complex scientific findings and measures of benefit and harm. Needs assessment is effectively coordinated with other EHC activities.
c. Report of needs assessment is clear, complete and useful; identifies specific needs and issues/barriers for each audience; and recommends appropriate products.
d. Needs assessment and report of results are completed on timely basis, in accordance with Schedule of Deliverables.
e. Contractor is responsive to PO comments and suggested changes on drafts of needs assessment.
f. Contractor effectively updates needs assessment, as requested by PO.
g. Evaluation plan is clear, rigorous and useful for measuring impact of the various translational products on target audiences. Evaluation plan includes: (1) evaluation goals; (2) evaluation questions to be addressed; (3) measures for addressing questions; (4) data to be gathered; (5) sources of data; (6) data collection approaches; (7) types of analyses to be performed; (8) potential evaluation issues and methods to resolve them; and (9) timeline for major evaluation activities.
h. Evaluation plan contains both process measures (e.g., number of products requested or downloaded from Publications Clearinghouse, as well as intermediate measures (e.g., change in user knowledge or attitudes, usefulness of product in decision making process). Evaluation plan provides for exploration into impact of different dissemination modes.
i. Draft evaluation plan is submitted on timely basis, in accordance with Schedule of Deliverables.
j. Contractor is responsive to PO comments and suggested changes on draft evaluation plan, and effectively incorporates these in final evaluation plan.
k. Final evaluation plan is submitted on timely basis, in accordance with Schedule of Deliverables.
l. Contractor implements evaluation plan in efficient and effective manner. Contractor collects data and generates reports that are consistent with evaluation plan, and are useful in meeting evaluation goals.
Review of deliverables by AHRQ PO and other Government staff.
3. Planning, Design and Development of Information Products. a. Background information and plan for translation products is clear, complete and useful; and reflects (1) participation in kick-off organizational calls of projects resulting in EHC reports; (2) participation in Technical Expert Panel/Group calls; and (3) acquisition and review of relevant contextual information through literature review and key informant interviews, as necessary.
b. Background information and plan for products is submitted on timely basis, in accordance with Schedule of Deliverables.
c. Contractor develops clear, efficient and effective process for verifying the accuracy of messages and subsequent translation of messages with the CER authors and AHRQ.
d. Contractor assembles a highly qualified team of content experts to review CERs.
e. Contractor completes reviews of and develops key messages for each CER or research report provided by AHRQ.
f. Key messages developed from CERs and research reports are accurate, clear, relevant, and useful to target audiences, and provide a sound contextual background for understanding and using the messages. Messages are stated for each audience for which products will be developed.
g. Key messages are developed on timely basis, in accordance with Schedule of Deliverables.
h. Contractor is responsive to PO comments and suggestions for changes in key messages and supporting documentation.
i. Development and assessment criteria for information products are clear, easy to use and well-supported by current tools and research.
j. Contractor identifies at least five individuals to review the draft criteria, whose qualifications demonstrate expertise in translation of complex heath information for various audiences.
k. Development and assessment criteria are developed on timely basis, in accordance with Schedule of Deliverables. Contractor is responsive to comments and suggested changes from experts and AHRQ.
l. Detailed protocol for developing translation products is clear, complete and useful; and includes, but is not limited to, routine processes such as literature review, expert interviewing, focus groups or cognitive interviewing. Contractor is responsive to AHRQ comments and suggested changes in protocol.
m. Contractor develops an information product for each assigned topic and each audience (patient, provider, policymaker), as appropriate.
n. Information products are developed that effectively reflect approved key messages, audience needs, and factors that impact health decisions. Products are concise, use plain language, and are useful to the target audience(s). Products conform to AHRQ, DHHS and GPO policies, regulations and guidance on format, use of color and font, and presentation.
o. Draft products are submitted on timely basis, in accordance with Schedule of Deliverables.
p. Contractor assembles small group of outside experts on product scientific accuracy, who are able to review products within an abbreviated timeframe. Contractor promptly revises products based on expert review.
q. Contractor conducts pre-test that is effective in assessing audience comprehension, acceptability and appropriateness, and ability to implement in the respective setting. Revisions to products are completed on timely basis, and appropriately reflect results of pretest.
r. Final products (in "camera-ready" and in electronic file) are submitted on timely basis, in accordance with Schedule of Deliverables.
s. Contractor is responsive to AHRQ and experts' comments and suggested changes on draft information products.
t. Contractor submits a detailed protocol on the development of the specific products that is clear, complete and useful; and includes references to relevant protocols and documents used in development of the products. Development protocol is submitted to on timely basis, in accordance with Schedule of Deliverables.
u. Assessment of existing decision aids and related literature is clear, complete and useful; and describes what exists currently, key needs for future decision aids, and critical components of a targeted decision aid.
v. Assessment of existing decision aids is completed on timely basis, in accordance with Schedule of Deliverables.
w. Review criteria for selecting topics for decision aids are clear, concise and effective. Review criteria are submitted on timely basis, in accordance with Schedule of Deliverables.
x. Contractor convenes working group to develop a specific development approach for decision aids, and develops a generic protocol that is clear, complete and useful. Protocol is submitted on timely basis, in accordance with Schedule of Deliverables.
y. Proposal for decision aid development is clear and useful; and indicates: (1) importance of decision aid for consumer decision making; (2) content and methodological experts who will be working on decision aid; (3) overview of proposed decision aid (e.g. major categories of information provided, type of value clarification exercises); (4) data sources, particularly those required outside of the CER; (5) mechanism for ensuring that decision aid will conform to IPDAS standards; (6) proposed evaluation of decision aid; and (7) estimated timeline and budget.
z. Proposal for decision aid(s) is submitted on timely basis, in accordance with Schedule of Deliverables.
aa. Algorithms for decision aid(s) are clear, user-friendly, scientifically sound, and useful for the target audience. Decision aid(s) include, as a minimum, information to be conveyed in the decision aid, any needed tailoring of information, any preference elicitation or values clarification activities, and any guidance or concrete actions to be recommended by the decision aid(s).
bb. Contractor is responsive to AHRQ comments and suggested changes in decision aid(s), including review criteria, protocol, proposal and actual aids.
cc. Contractor completes development of decision aid and places the aid on the EHC website.
Review of deliverables by AHRQ PO and other Government staff.
4 Development and Implementation of Symposiums, White Papers and Newsletter on Health Care Decision and Communication Sciences. a. Plan for white paper series clear, complete and useful; and includes a theme, potential topics, authors, and timeline. Plan reflects scan of the literature, discussions with experts, and identification of knowledge gaps and emerging areas. Plan for white paper series is submitted on timely basis, in accordance with Schedule of Deliverables.
b. Contractor identifies authors, discussants and experts that offer experience and knowledge in health care decision and communication sciences. Lists of authors discussants and experts are submitted on timely basis, in accordance with Schedule of Deliverables.
c. Contractor provides efficient and effective facilitation and logistical support for meeting to discuss white papers. This includes agenda preparation, facilitation of discussions, meeting logistics (including meals, per diem and travel reimbursements), and preparation of a meeting summary. Meeting summary is clear, complete and useful; and includes topics discussed, gaps identified, and recommendations to address gaps.
d. Meeting to discuss white papers is held on timely basis, in accordance with Schedule of Deliverables.
e. Contractor is responsive to PO comments and suggestions on white paper series plan and proposed authors, discussants and experts.
f. Proposed topics, audiences, presenters and venues for health care decision symposiums support objectives of (1) reaching target audiences effectively; and (2) making maximum use of intermediaries. List of symposium topics, etc. is submitted on timely basis, in accordance with Schedule of Deliverables.
g. Contractor provides efficient and effective logistical support for symposiums, including: (1) contacting parties to discuss symposium plan; (2) completing paperwork required by the event sponsor; (3) notifying AHRQ upon symposium acceptance and event dates; (4) recruiting presenters; (5) assisting presenters with per diem and travel reimbursements; and (6) working closely with OCKT on marketing the symposiums.
h. Symposium report is clear, complete and useful; and describes symposium details, audience makeup, and symposium impact. Symposium report is submitted on timely basis, in accordance with Schedule of Deliverables.
i. Contractor is responsive to PO comments and suggested changes on symposium plans and reports.
j. Overview of newsletter effectively communicates range of potential topics, as well as target audience and general format. Overview is submitted on timely basis, in accordance with Schedule of Deliverables.
k. Newsletter is clear, well-organized and effective in: (1) broadly disseminating recent EHC activities; (2) highlighting research in decision or communication sciences that might impact translation of evidence-based information for use by decision makers; and (3) targeting individuals interested in comparative effectiveness and decision science research.
l. Draft newsletter is submitted on timely basis, in accordance with Schedule of Deliverables.
m. Contractor is responsive to PO comments and suggested changes on draft newsletter, and effectively incorporates these in final newsletter.
n. Final newsletter is submitted and posted on Web site on timely basis, in accordance with Schedule of Deliverables.

Review of deliverables by AHRQ PO and other Government staff.

Feedback of meeting participants.

5. Dissemination of EHC Program Products. (includes EHC Web site support) a. Contractor develops a clear and useful plan for updating the current glossary. Plan for updating current glossary is submitted on timely basis, in accordance with Schedule of Deliverables.
b. Contractor effectively updates current glossary, as appropriate, to capture new terms and phrases used in EHC Program and other relevant sources;
c. Contractor develops an effective process to make terms and phrases more understandable and meaningful, including ways to provide context.
d. Contractor is responsive to PO comments and suggestions for changes in on draft glossary updates, and effectively incorporates these in final glossary.
e. Contractor develops plan for development/update of user's guide that is clear, complete and useful; and effectively reflects literature review, contacts with relevant consumer and professional organizations, and, as appropriate, inputs of experts in shared decision making, evidence-based medicine, and communications.
f. Plan for development/update of user's guide is submitted on timely basis, in accordance with Schedule of Deliverables.
g. Contractor develop detailed protocol for development/update of user's guide that is clear, complete and useful; and includes procedures for any focus groups or cognitive interviews, interview guides, surveys, or other development materials, and an outline of key sections.
h. Development protocol for user's guide is submitted on timely basis, in accordance with Schedule of Deliverables.
i. Contractor develops/updates draft user's guide for general public that is well-organized, understandable, and useful to the general public in incorporating evidence into their health care decision making. Guide effectively helps users understand and apply medical evidence, whether from the EHC Program or other sources, within context of potential decisions.
j. Contractor submits draft user's guide on timely basis, in accordance with Schedule of Deliverables.
k. Contractor is responsive to PO and external reviewer comments and suggestions for changes in user's guide, and effectively incorporates these in final guide. Contractor submits final user's guide on timely basis, in accordance with Schedule of Deliverables.
l. Contractor works effectively on a collaborative basis with the Scientific Resource Center, public and private sector intermediaries, and OCKT in developing a dissemination and implementation plan for products developed under the contract.
m. Dissemination and implementation plan is clear, complete, and useful; and includes: (1) methods to increase accessibility of all products to targeted audiences and the public; (2) type of medium and distribution channels by product type and audience type; (3) methods to leverage intermediate distribution through public and private sector entities; (4) description of responsibilities for dissemination among DSC, OCKT, and intermediaries; and (5) timeline of dissemination activities.
n. Draft Dissemination and implementation plan is developed on timely basis, in accordance with Schedule of Deliverables.
o. Contractor is responsive to PO comments and suggested changes on draft implementation plan and effectively incorporates these into final plan.
p. Final Implementation plan is submitted on timely basis, in accordance with Schedule of Deliverables.
q. Contractor works effectively on collaborative basis with OCKT in implementing dissemination plan, concentrating on electronic and other non paper-based dissemination mediums. (Note: Primary responsibility for dissemination is held by DSC.)
r. Information products are widely disseminated and used by target audiences, consistent with the dissemination and implementation plan.
s. Contractor effectively coordinates with relevant AHRQ staff and staff from the first DSC and the Scientific Resource Center in transition of the Web site without interruption of Web site activities.
t. Contractor is responsive to PO comments and suggestions for Web site changes and improvements.
u. Web site efficiently and effectively supports the following objectives: (1) post products in various formats for vendors, patients, clinicians, and policy makers; (2) advertise events; (3) post white papers and decision algorithms; (4) serve as a dissemination mechanism; (5) host e-mail distribution lists; (6) be searchable; and (7) be versatile and expandable to accommodate varied needs of users while allowing for expansion and technological improvements, and technical and information environment changes; and (8) develop individual pages for each AHRQ project or program, in consultation with PO.
v. Web site efficiently and effectively supports Section 508 requirements, as defined in SOW.
w. Web site efficiently and effectively provides the following capabilities:
(1) Make site available to invited participants through unique URLs that are sub-ordinate to the AHRQ Home Page.
(2) Support information searches to locate information locally stored or remotely linked data, contain sections describing how contents are maintained, and provide search feature that supports keyword searches and scrolling for information.
(3) Contain direct link to a privacy statement approved by the PO.
(4) Provide support for Web production, provide all application hardware to host Web site and connect it to the Internet with sufficient bandwidth to meet project needs, and provide a physical location to house the Web site.
(5) Support a load tolerance of at least 100 simultaneous users and a minimum of 1,000 user sessions per day.
(6) Provide contingency plan in case traffic appears to be exceeding these estimates so that it will be able to reconstitute the hardware/software platform to tolerate increased load within 60 days of a noted need to expand.
x. Web site efficiently and effectively provides the following other capabilities:
(1) Provide mechanisms for 24 x 7 systems near-full time availability.
(2) Ensure that any hardware or software failure does not result in loss of more than 27 hours of current data, any software failure does not result in more than 72 hours of downtime, any hardware failure does not result in more than 72 hours of downtime, and any Internet connectivity failure does not result in more than 48 hours of downtime.
(3) Maximize security and data privacy (even in the event of a catastrophic failure) and use necessary and appropriate firewall mechanisms and/or other blocking devices.
(4) Report any unauthorized access immediately to PO.
(5) Comply with all Federal and DHHS security guidelines.
(6) Not require the provision of personal information to visit Web site, and personal information not automatically collected.
(7) Post the AHRQ and DHHS logo prominently on Web site.
(8) Use COTS software to track and monitor Web site usage, track keywords entered into search engine, and provide database capabilities.
y. Web site maintains Web-enabled versions of decision aids and related products that result from MMA related activities.
z. Web site efficiently and effectively serves as a repository and clearinghouse for program products, educates consumers about health care decision making, allows stakeholders to provide input into the program, and through news and e-mail distribution list sign-up, allows the public to be notified of new activities.

Review of deliverables by AHRQ PO and other Government staff.

Feedback of Web site users.


Return to Contents

Attachment 5 - AHRQ Application and System Development Requirements

AHRQ has implemented a Distributed Systems Engineering Lab (DSEL) to support all internal development efforts and provide the facility for housing the software and documentation for all AHRQ sponsored systems and applications, regardless of where the system or application is hosted.

AHRQ uses a System Development Lifecycle (SDLC) framework which is consistent with the HHS Enterprise Lifecycle Framework (EPLC).  This framework is the basis for implementation of the DSEL, conduct of development projects and the Rational Unified Process (RUP)/Capability Maturity Model (CMM) based processes that support its implementation. The SDLC framework provides a disciplined approach which employs the following traditional project phases:

  • Concept
  • Initiation
  • Planning
  • Requirements Analysis
  • Design
  • Development
  • Testing
  • Implementation / Deployment
  • Operations and Maintenance
  • Retirement

The AHRQ SDLC framework is closely aligned with the disciplines defined in the Rational Unified Process (RUP).  The IBM Rational Suite of tools has been adopted by the Agency to provide a standard IT development environment for AHRQ sponsored systems and application development projects.  The AHRQ SDLC framework has been enhanced through the use of tailored processes and artifacts based on the RUP methodology.  The documentation deliverables required for all Information Technology (IT) projects are based on specific RUP artifacts identified by AHRQ.  The Rational ClearCase libraries housed within the DSEL provide the repository for housing software and documentation artifacts related to all AHRQ sponsored systems and applications, regardless of where the system or application is hosted.

Contractors are not required to follow the RUP development methodology or use the Rational Suite of tools; however, the Contractor's SDLC must be capable of producing AHRQ required system deliverables containing the required content as described further in the following section.  It is required that the Contractor use the lifecycle phases defined in the AHRQ SDLC framework and obtain PO approval before moving from one phase to another.

This approval process corresponds to the stage gates in the HHS EPLC model. The contractor must also conform to AHRQ Configuration Management (CM) and change control standards which require appropriate controls for managing software and documentation baselines, changes to software artifacts using an appropriate IDE or version management tool, document change requests and obtaining approval through a formal change control process that requires Project Officer (PO) and possible AHRQ IT approval prior to implementation.

The following table identifies the documentation deliverables required for all IT projects and the content required for each deliverable.

Deliverable AHRQ Life Cycle Phase Formats
Project Initiation Document Project Initiation MS Word
Project Work Plan Project Planning MS Project
System Requirements Document (SRD) Requirements and Analysis Rational Requisite Pro, MS Word
Requirements Traceability Matrix Requirements and Analysis Rational Requisite Pro, MS Word
System Design Document (SDD) Software Design Rational Software Modeler, MS Word, Rational Software Architect
Test Plan Testing MS Word
Test Scripts Testing MS Word, Rational Test Manager
User Acceptance Testing Report Implementation MS Word
User Guide Deployment MS Word
Operations Manual Deployment MS Word
Version Description Document Deployment MS Word

System Documentation

The Contractor will provide to the Agency system documentation of all proposed hardware, software, security, backup/recovery, and other information technology infrastructure and components and solutions needed to support this project.  The documentation is to be delivered to the Project Officer for review and approval for each release.  This documentation will be provided according to the content standards specified by AHRQ and will be maintained in the Agency's Rational ClearCase Repository as a unique project library created and maintained by the AHRQ CM Manager.  All documentation will be baselined with each system release. In addition, the source code for each production release will be delivered and stored in the same project library as the documentation artifacts. The contractor will be required to update these baselined artifacts for each production release of the system.  Sample documents and templates for the required documentation artifacts are available as guidance.  The following documents as mentioned in Table 1.1, "Documentation Deliverables", are required by AHRQ.

Project Initiation Document

The Project Initiation Document (PID) is intended to be a statement of purpose and scope for initiating a given project and a guide to manage expectations in both process and deliverables throughout the System Development Lifecycle (SDLC).  The PID defines the business case for the project by defining the purpose, the milestones, resources, objectives, costs, risks including mitigation strategies, and the artifacts and IT technologies (architecture) utilized and produced for, and during, the project. The PID serves as the formal funding commitment document approved by the COTR and Stakeholders.  Additionally, the PID must be approved by AHRQ IT management, and in some cases, the AHRQ Information Technology Review Board (ITRB) for technical viability, adherence to Agency Enterprise Architecture (EA); technical standards and formal Project Management requirements as derived from Departmental standards and accepted Project Management Institute (PMI) Project Management Body of Knowledge (PMBOK) standards.  In the case of external development contracts, the PID can be satisfied by the formal proposal submitted by the vendor and accepted by AHRQ. 

Project Work Plan

The System Project Plan or Project Work Plan (PWP) provides a method to assign and track the project resources, hours and specific deliverables. This plan provides the detailed Work Breakdown Structure (WBS) and resource loading that can be used to identify project costs and is intended for the project manager to track the schedule and cost of a project, including development of Earned Value Management (EVM) measures. The PWP is delineated by the phases of the project which include Project Initiation, Generation of the PWP Schedule, Requirements Gathering, System Design, System Development, System Testing including User Acceptance, System Deployment and System Support and production of project deliverables which require COTR or Stakeholder acceptance and signoff to continue project tasks identified in the PWP. 

System Requirements Document

The System Requirements Document (SRD) contains the system requirements, use cases and supplementary specifications that provide the basis for design and development of the system.  The following information is provided for each requirement identified in the document:

  • Requirement ID, Name and Title
  • Requirement Description
  • Software Release Version
  • Use Case Model
  • Use Case Specifications
  • Supplementary Specifications

A text-based Functional Requirements Document may be provided instead of a Use Case Model, Use Case Specifications, and Supplementary Specifications.

Requirements Traceability Matrix

The Requirements Traceability Matrix (RTM) associates requirements with the work products that satisfy them.  This matrix is created at the beginning of a project's lifecycle to trace the requirements from identification through testing.  The project elements are traced as they relate to other project elements, especially those related to requirements. 

The purpose of establishing traceability is to help understand the sources of requirements, manage the scope of the project, manage changes to requirements; assess the project impact of a change in a requirement; and verify that all requirements of the system are fulfilled by the implementation. 

The following values are required for the traceability matrix:

  • Requirement ID and Title;
  • The version of the system in which the requirement will be implemented;
  • The Use Case to which the requirement can be traced;
  • The version of the design document in which the requirement is implemented;
  • The ID of the test script in which the requirement is tested;
  • The version number of the source code in which the requirement is implemented.

The figure below shows a sample of the data traced through a project's life cycle.

Sample of data traced through a project life cycle.  Table with columns labeled: Requirements, Version, Trace to UC, Trace to Text, Trace to Source, CR, and Status. Contains eight rows of sample data.

System Design Document

The System Design Document (SDD) details the design and implementation of all custom software features of the system. The design descriptions must include use cases that detail the interaction which occurs between a user and the system.

The document describes the general nature of the system, and describes the architecturally significant parts of the design model, such as its decomposition into subsystems and packages.  For each significant package, a section of the document should detail its decomposition into classes and class utilities.  Architecturally significant classes should be introduced and a description of their responsibilities should accompany the introduction.  Any significant relationships, operations, and attributes should be detailed in this document.

The document should be organized by use case, so that it provides traceability back to the initial requirements. The document must also contain a description of the database model and data elements used to support the application. This data can be referenced to an appropriately maintained Entity Relationship Diagram (ERD) and data definitions which conform to CM standards and are appropriately maintained in the Rational CM Libraries.

Test Plan

The purpose of the Test Plan is to define the approach for testing a particular application or system.  The Test Plan is a high level description of the testing process which will be performed. The Test Plan outlines the types of testing to be performed, the requirements to be tested, the test environment, testing tools, pass/fail criteria and a risk assessment. At a minimum the document should contain the following:

  1. Test Description
    • A general overview of the plan for testing the entire system.
    • Test objectives for all testing levels (e.g. module, unit).
    • Scope and guiding principles for the testing effort.
    • A policy for resolving conflicts that arise during the testing process
  2. Acceptance Criteria
    • The criteria agreed upon with the customer for acceptance of the software.
  3. Approach
    • How each major group of software features will be adequately tested.
    • Major testing activities, techniques, and testing tools.
    • Test Environment – Hardware, Network, Software and Test Database
  4. Tasks
    • The individual tasks that must be performed.
    • The individual or organization responsible for each task.
  5. Schedule, Resources & Milestones

Test Scripts

The Test Scripts define testing scenarios completed for an application.  Each scenario details the steps to be performed, expected results and pass/fail criteria.  At a minimum the document should contain the following:

  • Test Script Identifier
  • Test Description
  • Test Objective
  • Test Environment/Setup including any required data such as Login credentials, etc.
  • Mapping to specific requirements and design elements contained in the SRD and SDD
  • Step sequences and actions
  • Expected Results
  • Pass/Fail Criteria
  • Actual Results
  • Comments

User Acceptance Test Report

The User Acceptance Testing (UAT) Report should include a summary of the testing environment (hardware, software, tools, participant list, etc.) and procedures used to demonstrate and obtain stakeholder approval of the application or system prior to production deployment. The UAT Report should contain a mapping to the SRD and SDD items included in the release as well as an exception list or identified change requests that were generated as the result of testing.

User Guide

The User Guide is completed prior to production.  The User Guide is a "How To" manual which navigates the user in detail through the use of the application.  This document usually contains system screen shots and provides step by step instructions for completing tasks and activities.  It is written on a business level with the needs of the user in mind.  At a minimum the document should contain the following content.

  • Introduction
  • Summary of the application
  • Glossary (Definitions/Acronyms)
  • Procedures (Step-by Step instructions on how to use the system)
  • Troubleshooting tips

Operations Manual

The Operations Manual provides guidance and defines procedures related to the operational implementation of the system. At a minimum, the document should contain the following:

  • System Overview
  • Statement of acceptable use of the system and information
  • Hardware and software descriptions
  • Interfaces with other Systems and Databases
  • Access and authentication requirements
  • System Configuration and Administration Procedures
  • Security procedures including virus protection
  • Incident Reporting and Handling
  • System Startup and Recovery Procedures
  • Change Management Procedures

Version Description Document

The Version Description Document (VDD) identifies and describes the general release information, and inventory of software released (Bill of Materials), for a specific application, including prototype iterations. The document should include the following sections listed below:

  • Introduction - Describes the objective of the document, defines the release identification and provides contact information.
  • General Release Information - Provide information about the specific release, including any interfaces and dependencies
  • Installation Instructions - Describes the steps required to install the software.
  • Version Description - Provides an inventory of List Objects and Module Types such as: class files, SQL Scripts, HTML files, DTD and XML files.
  • Recovery Instructions - Describe the steps required to reconstruct the release from the product baselines, established in the configuration management library.

Web Product and Web Site Development Guidelines

The following list highlights basic issues that need to be addressed when developing Web tools or sites under contract that will be publicly available when launched to ensure deliverables are on target, in compliance with legal and policy requirements, and do not require expensive rework to meet Federal and Department of Health and Human Services requirements for information resources. 

Guidelines for Web-Based Products

Retrofitting Web-based products after the fact is highly undesirable because it adds time and costs to the process of making these products publicly available. All products that are developed with the intent of being posted on the AHRQ Web site should meet the following minimum requirements:

Titles of Products

Coordinate with your project officer on the titles of your products. They need to be concise and relevant to the purpose of the project, but cannot include the name of the contractor or grantee as the performing organization as part of the title. Report titles should be no more than 10-words maximum and Web-based tools should be no more than 5-words maximum (make every word count—eliminate initial articles such as "The" or "A" ). Titles need to be distinct enough to differentiate among similar sounding products.

Quality Control/Editorial Review

This involves checking for spelling and grammar mistakes, formatting issues, general consistency, and style.  This should be done by the AHRQ grantee or contractor prior to submission of the final product for posting on the AHRQ Web site. Federal resources follow the GPO Style Manual which is available electronically at:
http://www.gpoaccess.gov/stylemanual/browse.html

Accessibility

As an agency of the Federal Government, AHRQ must ensure that anything that is posted on our Web site is in compliance with requirements for information resources under Section 508 of the Rehabilitation Act.  Also, federally funded resources need to be generally available to users in multiple formats to ensure that we are not forcing a particular platform, operational system, or proprietary software package on users.

Intellectual Property Rights

Before we can post a product on the AHRQ Web site, we must have a written explanation of the following four questions:

  • Who retains the copyright?
  • Who has licenses for what purposes and uses?
  • What are the constraints imposed?
  • Who grants permission for further use or adoption?

Usability

Web resources should include usability testing, evaluation, and modification as an integral and recurring part of the development effort to ensure they are effective for the electronic business processes they are designed to facilitate. A set of Research-Based Web Design and Usability Guidelines that should be consulted are available at: http://www.usability.gov/guidelines/index.html

Beta testing prior to release is desirable, evaluating the product against usability heuristics. As feedback is received and products are updated, the revisions will need to be designated by version number and date of release.

Privacy Act Protections

Web resources are subject to the Privacy Act and this can impact both the development of Web-based tools and the users of those tools. Persistent cookies should not be programmed into the functionality of a Web-based tool, although session cookies are allowed. Registration for use cannot be requested if this would involve collection of individual identifiers from the users. Although exemptions to both rules can be sought, this involves a strong justification and several levels of review for approval through the U.S. Department of Health and Human Services (HHS).

Guidelines for Web Sites

Web sites being supported through contracts are considered Federal information resources and as such are required to be in compliance with laws, policies, and directives that affect such resources.

This includes content management and information categorization, including standard metadata, under the E-Gov Act requirements and Office of Management and Budget issuances to Federal agencies on IT resources.

For recommendations and guidance on requirements and best practices, go to:  http://www.usa.gov/webcontent/reqs_bestpractices/best_practices.shtml

Clearance

Web resources require clearance by HHS--including justification against a set of criteria. Publications cleared for printing are cleared for Web uploading at the same time. Web resources must comply with the numerous laws and directives that affect federally funded electronic information resources. Web content loaded on a site by contractors must be appropriate and follow all laws and directives. AHRQ Offices and Centers must coordinate initial review through AHRQ's Office of Communications and Knowledge Transfer (OCKT) before launch, and OCKT will coordinate departmental clearance.

Domain Names

All domain names for any Web resource funded in whole or in part by Federal funds must be registered as .gov through HHS with the General Services Administration (GSA). Although other domains, such as .org, .net, .edu, .com may also be reserved by the Agency, the .gov domain must be registered and that domain name will need to be indexed by USA.gov, the GSA portal to government-funded resources. The USA.gov link is then required on the home page of the site. Coordinate with OCKT on domain name requests.

Editorial Review

All content for upload needs to be reviewed to ensure consistency and compliance with best practices and established style and conventions. As a minimum, the copy needs to be production edited to ensure there are no typos and the GPO Style Manual is followed for punctuation, spelling, use of numerals, abbreviations, etc. Do not use unexplained acronyms; they need to be spelled out on first reference in any document or file. There should not be anything marked DRAFT on a public site. Once the materials are uploaded, they are published and considered in the "public domain." Do not use placeholders for content that does not exist. Government funded sites should not have anything designated "under construction." A process needs to be established for regular review of content and updating. Additional materials need to undergo editorial review and be approved before uploading. The GPO Style Guide is available electronically as a reference at: http://www.gpoaccess.gov/stylemanual/browse.html

Accessibility

Under the Rehabilitation Act, Federal agencies have an obligation to provide equal access for the disabled to their information and services. Requirements are specified in section 504 for individual accommodation and more recently in section 508 for electronic and information technology, which includes Web sites and multimedia products. Equivalent alternatives are required for auditory and visual information, such as providing alternative descriptive text for images for the blind and providing captions for audio-video files for the deaf. Written transcripts are required for all streaming audio. PDF files can be offered in conjunction with accessible files, such at HTML versions, but avoid uploading PDF-only versions of documents unless they are fully accessible PDF formats. OCKT has software used to evaluate Web sites and can provide a report on any accessibility violations that would need to be addressed before launch. Specific requirements are available at: http://www.section508.gov

Privacy

A privacy policy notice must be prominently displayed, and the Web site host has to follow it. A machine-readable format (P3P) of the privacy policy notice must also be uploaded to the site. A Privacy Impact Assessment is conducted to determine what kind of personal information is contained within a system, what is done with that information, and how that information is protected.  Sites are periodically audited to ensure that they observe their stated privacy policy. A Privacy Act System notice may need to be prepared and published for users to register on a site if the registrations represent a group of records, under the control of the Agency (or a contractor), that can be retrieved by personal identifier. This notice must go through several levels of review--including the Office of General Counsel--and be published in the Federal Register. Persistent cookies cannot be used on Federal sites unless the Secretary of HHS grants an exemption, and this involves a strong justification and review process.

Web Site Mailbox

Every Web site must provide full contact information for the sponsor and have a Contact Us link for submission of comments or questions as a customer feedback mechanism. Web site e-mail is subject to the same privacy and records management issues that affect the overall Web site as well as departmental standards for handling inquiries and customer feedback. Each Web site must provide relevant Frequently Asked Questions that are included in the customer relationship management system used to handle AHRQ Web site inquiries.

Records Management

All content on the site and e-mail generated by the site must be archived electronically and handled according to records retention schedules and disposition authorities as established with the National Archives and Record Administration. This requirement also affects Web site log files and statistical reporting on Web site usage. For guidance on requirements, go to:
http://www.archives.gov/records-mgmt/policy/managing-web-records-index.html

Information Collection Budget

If a Web site is used to collect information from users, such as for surveys, evaluations, or beta testing feedback, then the Office of Management and Budget must first approve the burden hours for such an effort for this collection. A notice must be posted on the Web site at the point of collection with the OMB approval number and a statement on the process of collection. 

Intellectual Property

Copyright and trademark protections need to be observed on Web sites. Permissions for use must be granted for any copyrighted information included and registered trademarks need to be reflected in copy. Any copyright or trademark constraints related to materials uploaded to a site must be specified for users. Public domain does not extend outside the borders of the United States. Therefore, foreign countries must request specific permission for use. Given the global nature of the Internet, citation as to source is a critical issue.

Linking

External links constitute an implied endorsement and create a business advantage for the linked sites. OMB requires Agencies to do a risk assessment of external links, and potential links need to be assessed against the HHS and AHRQ linking policies and criteria. If a site deviates from these policies, then the specific review and selection criteria must be justified and posted on the Web site for full disclosure. Outside Web resources may link to Agency resources providing that the link is not displayed in any way that would imply an endorsement by the Agency of a specific commercial product or service.

Electronic FOIA

The Agency is required by law to have an electronic FOIA reading room and to provide materials that can be requested under the Freedom of Information Act in electronic form, if so requested. HHS requires that any Web resource funded by the Agency provide a link to the AHRQ Freedom of Information Act page on the main AHRQ Web site.

Usability

Web resources should include usability testing, evaluation, and modification as an integral and recurring part of the development effort to ensure they are effective for the electronic business processes they are supposed to facilitate. For best practices in initial development or redesign of Web resources, go to: http://www.usability.gov
 
Web Sponsor Identity

AHRQ has uniform principles to identify AHRQ as the primary sponsor of AHRQ-related Web sites. These principles reflect HHS best practices for a consistent look and feel of Web resources, reinforce credibility, and support HHS and Agency branding efforts. The four specific principles that should be consistent across all AHRQ-funded Web sites are:

  • Web site URL name: The name of a Web site should always contain AHRQ in the URL. A Web resource should either be a folder on the main AHRQ Web site (www.ahrq.gov/chiri) or a third-level domain of the Web site (www.webmm.ahrq.gov).
  • Title of Web site project: AHRQ's name should be part of the formal title and appear at the beginning of the Web site's project name when referenced in print or promotional materials. For example: AHRQ's Web Morbidity and Mortality online journal.
  • HHS and AHRQ logos: The HHS and AHRQ logos should be featured prominently on the Web site and in materials that are used to market that Web site.
  • Web site home page format: The Web site home page should have common design and navigation elements with the HHS Portal and the AHRQ Web site so that all Web sites look as though they belong to the Department and AHRQ Web family. All AHRQ domain sites must include a standard banner and footer that are branded for Web resources. Technical specifications and templates for developers to consult when designing Web resources are provided by the AHRQ Web Manager.

Return to Contents
Proceed to Next Section (Attachment 6)

Current as of May 2008

AHRQ Advancing Excellence in Health Care