Skip Navigation Archive: U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archive print banner

Centers for Education and Research on Therapeutics (CERTs)

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

Technical Assistance Conference Call: Summary

November 9, 2006

CERTs is a peer-reviewed program that conducts state-of-the-art research to increase awareness of new uses of drugs, biological products, and devices; ways to improve the effective use of drugs, biological products, and devices; risks of new uses; and risks of combinations of drugs and biological products.

On November 9, 2006, the Agency for Healthcare Research and Quality (AHRQ) held a conference call to provide potential applicants with background information and to respond to questions about the preparation of applications in response to CERTs Funding Opportunity Announcements (FOAs) . The conference call is summarized below.

Note: PowerPoint® files can be viewed with Microsoft® PowerPoint® or with a free PowerPoint® Viewer.


Slide Presentation
Responses to Presubmitted Questions
        General Advice
        Questions About Research Centers and Themes
        Health Information Technology Research Center
        Coordinating Center
        Eligibility and Cross-funding Opportunity Announcement Questions
        Currently Funded CERTs
        Review Issues
        Electronic Submissions
Open Question Period for Conference Call Participants


The CERTs program began with the release of RFA HS-99-004 in 1999 and the subsequent award of grants pursuant to the Food and Drug Administration (FDA) Modernization Act of 1997. The legislation authorized the Agency that is now AHRQ to carry out a demonstration program with the FDA to conduct research and provide objective information on drugs, biologics, and devices. AHRQ continues to manage this program in coordination with the FDA under the Agency's reauthorizing legislation of 1999, Public Law 106-129.

The CERTs program was created in response to a legislative mandate to conduct state-of-the-art research to increase awareness of:

  • New uses of drugs, biological products, and devices.
  • Ways to improve the effective use of drugs, biological products, and devices.
  • Risks of new uses and risks of combinations of drugs and biological products.

The CERTs provide objective clinical information to:

  • Health care practitioners and other providers of health care goods or services.
  • Pharmacists, pharmacy benefit managers, and purchasers.
  • Health maintenance organizations and other managed health care organizations.
  • Health care insurers and government agencies.
  • Patients and consumers.

The CERTs investigate ways to improve the quality of health care while reducing its cost through an increase in the appropriate use of drugs, biological products, and devices and the prevention of adverse effects of drugs, biological products, and devices. The CERTs also conduct research on the comparative effectiveness, cost-effectiveness, and safety of drugs, biological products, and devices.

The purpose of the CERTs FOA's Research Centers, Coordinating Center, Health information technology addendum,  and budget submission instructions are to:

  • Solicit new CERTs Research Centers, including one on health information technology (IT).
  • Invite applications from nonprofit and for-profit organizations to act as the Coordinating Center for all CERTs Research Centers and the CERTs Steering Committee.

Return to Contents


Welcome everyone to the technical assistance call on the Centers for Education and Research on Therapeutics funding opportunity announcements. We're very pleased to have you joining us today.  I am Anne Trontell.  The people around the table at AHRQ are:

  • Michael Handrigan, CERTs Project Officer.
  • Michelle Burr, Grants Management Specialist.
  • Carmen Kelly, Pharmacist and AHRQ project officer of the current CERTs Coordinating Center.
  • Scott Smith, Director of AHRQ's Pharmaceutical Outcomes Portfolio.
  • Carl Ohata, Scientific Review Administrator.
  • Gurvaneet Randhawa, CERTs Project Officer.
  • Art Sedrakyan, CERTs Project Officer.
  • Skip Moyer, Special Assistant to AHRQ's Deputy Director.
  • Jon White, Manager, AHRQ Health IT Portfolio.

Return to Contents

Slide Presentation

Those of you who registered for the call received a copy of the slide set (PowerPoint® file, 690 KB; Text Version).         

Slides 1 and 2:  AHRQ's mission is to improve the quality, safety, efficiency, and effectiveness of health care for all Americans.

Slide 3: The CERTs program is a research demonstration program funded as a U18 cooperative agreement by AHRQ in partnership with the FDA. It focuses on research and education on therapeutic products, by which we mean drugs, medical devices, and biologics already marketed in the United States.  The CERTs also work with other agencies in the Department of Health and Human Services that are involved with therapeutics. This may include not only FDA, but also the Centers for Medicare & Medicaid Services (CMS).

Slide 4:  By way of CERTs legislative history, the Food and Drug Modernization Act authorized the CERTs program, which has since been incorporated into AHRQ's reauthorization.  The CERTs program is a permanent program at the Agency.  The CERTs legislation calls for a cooperative agreement mechanism, which is a grant mechanism that allows Government involvement but not the level of extensive control that occurs with contracts.  In addition to their support from AHRQ for infrastructure and pilot projects, the CERTs leverage their individual and combined expertise through public or private partnerships under the oversight of the CERTs Steering Committee.

Slide 5:  This slide shows the many rich interrelationships between the CERTs Research Centers, the Coordinating Center, the Steering Committee (which is comprised of the principal investigators of the Research Centers and other members), a wide variety of public agencies, and private industry and foundations.  The Coordinating Center is a vital communication link among all these various organizations.

Slide 6: The CERTs vision is to be a trusted national resource for people seeking to improve their health through the best use of medical therapy.  Its mission is to conduct research and provide education that advances the optimal use of drugs, medical devices, and biological products.

Slides 7 and 8: The links to the funding opportunity announcements, including the addendum for the Health Information Technology Research Center, are displayed here along with the announcements for the technical assistance call and budget submission instructions. 

The last item is a recent posting that will be important to consider in your applications.

Slide 9: By way of orientation to the Health IT Research Center, it is open to all who are eligible to apply the CERTs Research Centers FOA or the Coordinating Center FOA, as well as to CERTs Research Centers initially funded in 2006. The Health IT Research Center is described in the addendum. Its objectives are to develop, demonstrate, and disseminate evidence and educational/translation materials on the role and use of health IT tools in the safe and effective use of therapeutics. The FOA lists eight bulleted items that are potential areas of study.

Slide 10:  This slide shows the plans and current configuration of the CERTs.  There are currently 11 funded CERTs Research Centers, a Coordinating Center, and a Steering Committee.  Seven of these centers are in their final year of funding and are open for competition.  With the addition of dedicated funding for a CERTs research Center on health IT, a total of $8 million per year is planned, with up to $1 million per year for each Research Center, for a total of 4 years.

The CERTs network's Coordinating Center, which is up for competition, has anticipated funding of $400,000 annually for 4 years.  Four CERTs Research Centers were funded this year and are anticipated to continue for another 4 years. Their names and associated themes are:

  • Iowa (with the theme of aging).
  • Rutgers (with the theme of mental health).
  • Cornell (with the theme of devices).
  • MD Anderson/Baylor (on patient adherence and consumer education).

Slide 11:  This slide describes a center. There is certainly flexibility in what a center will look like. They are similar in that AHRQ funding supports infrastructure and the conduct of three to five proposed projects.  The Centers, as I stated earlier, support the mission of multiple Health and Human Services agencies of the Department of Health and Human Services (DHHS).  They may modify projects to support programmatic needs that are informed by AHRQ and FDA.  The Centers are to have identifiable and quantifiable program goals based on patient outcomes and are to develop outreach to improve local and national health outcomes through partnerships.  Themes are to be determined by the applicants in consideration of a number of factors that are described in depth in the FOA and that we'll discuss further later in this call in the context of the questions we've received.

Slide 12:  Let me conclude the slide presentation by reminding you of key dates. 

  • The next key date is the date for the receipt of letters of intent; that is November 10. These are requested but not mandatory. 
  • The date of December 14 for receipt of applications is mandatory. All applications are by electronic submission. I urge those of you who have not used this system to initiate registration as quickly as possible. 
  • We anticipate the peer review date to be sometime in March 2007.
  • The earliest anticipated start date would be July 2007.

Return to Contents

Responses to Presubmitted Questions

During this segment, we will read the questions and answers that we've received so far on the FOAs. We have anonymized those that reveal individual plans that might be more appropriate to discuss offline.  After we have gone through these questions, we'll open the floor to your questions and we'll take them in the following order:

  • Research Centers including the Health IT Center.
  • Coordinating Center questions.
  • Questions about budgeting and grants management.
  • Questions about grants review.
  • Budgeting and Grants Management.
  • Grants Review.

General advice

We start by offering some general advice. The first is really a disclaimer to alert you to our intent in this call to assist you in finding and understanding the information in the FOA.  It's important to bear in mind that the information in the FOA is considered to be final and definitive. It's really where all the answers lie, and if what we say today in any way appears to contradict the FOA, the FOA is the authority. 

A useful hint for those of you who view these documents electronically is to use the Ctrl-F function to search for key words that might be relevant to your question.  Should the FOA be silent on your questions, look to the electronic 424 Grant Application Form, which draws upon PHS Form 398.  There are many links available to help assist you if this is your first electronic submission.  The registration process may take as long as 4 weeks, so we urge you to initiate registration if you have not already done so.

In instances where the FOA or the various application forms are silent, we will give you our best knowledge or opinion.  However, it's important to recognize that if our opinion is in conflict with the documents you have, then the documents are right and we are wrong.

We alert you to the language in the FOA.  Where the word "must" is used, that in fact is a factor that will be used in determining whether or not an application is responsive. When we use the word "should," that means it's a matter for the review panel to consider.  We direct you to look closely at Section 5, application review information. In particular, Sections V.2 and V.2.A offer insights on how applications will be reviewed and scored, as well as considered for funding.  Recall the link to instructions for budget submissions which were recently published in the NIH Guide.

Questions about Research Centers and Themes

Question: One question asked what themes were in the scope of the FOA.

Answer:  The theme must be relevant to therapeutics, which are defined to include drugs, devices, and biological agents.

Question: Several questions concern how to determine whether or not a chosen theme will be considered adequate.  Others have asked whether some of the themes suggested in the FOA might be used in part or in whole.

Answer:  The language in the FOA states that the theme should be broadly based in its focus, such as a particular setting or population, rather than any single disease or condition.  We expect that the Centers will select and document themes that have a high likelihood of potential impact on practice, and that applicants will have developed priorities and a strategic plan for comprehensive education and educational research in therapeutics within a chosen theme.  The chosen theme and associated projects should include a strong justification as to how the research and education will be translated into public health impact.

In FOA Section I, following the areas of potential themes, there's a paragraph that lists eight considerations that define AHRQ priority topic areas or conditions. In addition, in the Section V, Application Review Information, there's a description of review criteria that may be applicable to an applicant's proposed theme. Applicable criteria are described under the headings of Significance, Responsiveness, and Relevance to Legislative Mandate Funding decisions will consider multiple factors, including relevance to program priorities and program balance.

Question: Is there continued interest in having more work in the thematic areas of the seven CERTs that are now in their final year of funding?

Answer:  The FOA states that applicants are required to develop proposals that address gaps in the current CERTs portfolio.  The four CERT Research Centers that were funded in 2006, as well as their thematic areas, were already described in the slide presentation.  Applicants may choose themes from those Research Centers that are in their final year of funding. They may address other themes that are listed in the FOA or they may develop themes with the guidance of the criteria that I've just described.

Question: Are multiple sub-themes allowed?

Answer:  A sub-theme is described in the FOA as being of value to the entire CERTs program.  The FOA describes the option of an applicant having a sub-theme.  If additional sub-themes are proposed, they should be well justified and supportable within the proposed application.

Question: How important is it to capture multiple priority areas in the proposal as opposed to focusing on just one?

Answer:  AHRQ included these priority areas to help in choosing a theme for the Research center that will have a beneficial impact on public health.  They will be part of what the review panel will consider in evaluating the proposals for significance.

Question: Although we thought we had stated it clearly in the FOA, a number of questions concerned how many projects should be included in the application for the Research Centers.  We've also been asked how much detail should be included on these projects.

Answer:  The FOA states that funds should be budgeted to provide for staff support to undertake the design, development, and implementation of between three and five research and education projects.  Each proposed project within the theme should have a limited number (no more than two) of quantifiable, desired patient outcomes. The applicant should include quantifiable methods and data sources for measuring progress toward these goals, with the goals linked to a plan for implementation.

Question: What should we do for the future, or out years, of the application?  Is it possible to solicit pilot projects from institutions outside the institution making the application and to fund them with proposal funding?

Answer:  We remind you that AHRQ funding decisions are based upon the review panel's evaluation of the proposed projects in the application.  Grants management strongly discourages the use of award funds for projects that do not undergo a peer review process.

Question: Given that this is a 4-year grant cycle, the first year may involve gathering and synthesizing new information that will shape the research and education projects for Years 2-4.  How much detail do you want about proposed studies and educational interventions planned for Years 2-4?  How much detail do you want for the educational interventions?

Answer:  Proposed studies should provide sufficient information that the review panel can evaluate them for the review criteria listed in the FOA and previously described.


Question: Please describe in more depth the key elements you wish applicants to address in their discussion of the infrastructure for the grant. 

Answer:  The FOA indicates that the Research Center funding provides infrastructure support, which means a research team with adequate facilities, institutional support, and access to appropriate expertise needed to perform the research, education, dissemination, and translation of research into practice.  AHRQ funds are intended to provide infrastructure support for each Research Center to allow it to function effectively.  Funds should be budgeted to provide for staff support to undertake the design, development, and implementation of between three and five research and education projects.

The additional review criteria indicate that evidence should be presented of an infrastructure capable of maintaining the CERTs Research Center.


Question: Is there an opportunity for collaborative funding with partners, i.e., how much should be focused on using CERTs funds vs. other sources of funding)?

Answer:  Partnerships are a very important part of the current CERTs program.   A variety of Federal and private partners exist.  The CERTs Web site describes a number of these and also describes the principles by which these partnerships operate, as well as other relevant information.

Partnerships are also addressed in the FOA in several areas.  In particular, the applicant should identify and describe relationships with several intermediary and end-user groups. Intermediaries may include appropriate clinical specialty organizations.  End-user groups might be consumers or patients.  The applicant may consult with these intermediary and end-user groups on research products and implementation strategies as they are designed. The partners must be willing to help foster dissemination and translation of findings and products into practice. Letters of support from these organizations provide evidence of these partnerships.

Bullet 5 of Additional Guidance, under Part II, Section 1, Funding Opportunity Description, Research Objectives, describes considerations for partnerships.

We note that, for those projects where a nongovernmental partner has been identified, the applicant should include a description of the steps that have been taken to assure adherence to the principles set forth in the CERTs principles for public or private partnerships.  These principles can be viewed on the CERTs Web site.

Also, the review and selection criteria under the section on budget discuss the review considerations that are relevant to partnerships.

Question: Under the additional guidance bullet, AHRQ was asked for clarification about the statement: "Each Research Center should incorporate measures and goals of cost-effectiveness and efficiency into research and education activities."

Answer:  This additional guidance indicates that cost-effectiveness and efficiency considerations can inform the Research Center's choice of proposed research and education activities, and that the review panel may consider these as well.  Recall our previous general advice on what is described as "must" and what is described as "should."

Question: What should be the length of the research plan? 

Answer:  The length is the default of the SF424 Application Guidelines, which is 25 pages.

Health Information Technology Research Center

Question: Will the Health IT CERT be a standalone CERT with some partnerships? Will its main function be to support the other CERTs?

Answer:  The Health IT CERT will be a standalone Research Center like the other CERTs and will be encouraged to collaborate and form partnerships. Support of the CERTs Research Centers is a function of the Coordinating Center and is described in the FOA for the Coordinating Center. 

Question: Does the IT CERT consider information technology as a therapeutic device or as an adjunct to other therapies (e.g., translating research into practice or safety assessment)?

Answer:  In the majority of cases, health IT would be considered to be an adjunct to other therapies. However, we can imagine there might be cases where Health IT itself is the therapeutic device.

Question: Can U18 grant resources be used for software writing?   

Answer:  As long as it's not prohibited in the FOA, software writing costs can be included. Like all allowable costs, however, they must also be reasonable, allocable, and necessary.  A good budget justification is helpful.  Reviewers and funding decision-makers will look at requested costs in the context of the overall proposal.

Question: What was the impetus for adding the Health IT CERT?

Answer:  AHRQ has been doing a lot of therapeutic work in the health IT grant portfolio, and health IT is an increasingly important mechanism for delivering and gathering information.  There's also a desire to understand the Health IT systems themselves and how they can improve therapeutic delivery and coordination of care.  Funding for the Health IT CERT comes from AHRQ's health IT portfolio, which is an extension of AHRQ's patient safety initiative.

Question: Do current CERTs have any advantages compared to non-CERTs in competing for becoming the Health IT CERT?

Answer:  No. All applications will be reviewed by the same panel without preference or stratification by experience.  The health IT CERTs applications will be compared against each other for the final funding decision. The CERTs FOAs indicate that program balance (avoidance of duplication and overlap in the funding of Research centers) and geographic distribution, along with other factors, will be weighed in making funding decisions. 

Coordinating Center

Question: Section I: In the description of the Coordinating Center (CC) and the Principal Investigator (PI), AHRQ uses the terminology "a global program perspective."  Should that be taken literally to imply that the CERTs effort in the future will turn more international (i.e., global), or simply that the PI is expected to have a comprehensive view of the CERTs program?

Answer:  The words "a global program perspective" mean a comprehensive view of the CERTs program, looking at CERTs goals and objectives being fulfilled.  The FOA explicitly states that the focus of these programs is to increase the implementation of safe practice interventions in the United States.  For that reason, foreign institutions are not eligible to apply. No awards will be made to them.  However, foreign organizations may participate in projects as members of consortia or as subcontractors.

Question: Section III, Other-Special Eligibility Criteria/Special Requirements:In the bullet about educating users, does the ". need to develop a plan to . implement widespread dissemination to audiences such as State policymakers and clinicians..." imply that this is a new or expanded education and dissemination requirement?  Would this entail working directly with the User Liaison program of the Office of Communications and Knowledge Transfer?

Answer:  This bullet indicates a responsibility of the CC and emphasizes the need to further existing education and dissemination efforts of the CERTs program.  This includes supporting relevant subcommittees of the Steering Committee, which includes a cross-Center consortium of educational leads from each CERTs Research Center.  This is referred to in the FOA as the ESC, or Educational Subcommittee.  It will also involve working with AHRQ programs and AHRQ's Office of Communications and Knowledge Transfer to continue to develop plans for dissemination.

Question: Can AHRQ explain what "economies of scale" means in the context of the eighth bullet  under Section III, Other-Special Eligibility Criteria/Special Requirements?

Answer:  "Economies of scale" refer to the effective use of resources to look across all of the Center activities to find ways resources can be used most efficiently.  One example would be the use of combined meetings, such as "Think Tanks" or Steering Committee meetings, where two meetings are convened at the same location and adjacent time slots to minimize the need for traveling to separate meetings and to maximize participation.

Question: In the last bullet under Section III, Other-Special Eligibility Criteria/Special Requirements, on publications and "complete bibliography," can AHRQ clarify the meaning of this requirement? Whose publications should be included and please define "complete bibliography." How does this requirement relate to the publications that are included in the appendix?  

Answer:  The publications of the PI plus key personnel within the CC should be included with a complete listing in the form of a bibliography.  The bibliography refers to a complete listing of citations for articles that have been published.  The FOA indicates the appendix may include up to 10 publications, manuscripts accepted for publication, abstracts, patents, or other printed materials directly relevant to the proposed project.

Question: Section IV, Part 3.C, Use of CMS Data:  The CC would not itself (presumably) be doing projects that would require acquisition of CMS public-use data.  Therefore, can AHRQ clarify what should be discussed in the CC application about this issue?  The same general issues may arise about use of HCUP (Healthcare Cost and Utilization Project) and MEPS (Medical Expenditure Panel Survey) data.  These don't seem relevant for the CC responsibilities. 

Answer:  If an application for the CC does not contain plans for use of CMS data or other data resources, then this section does not apply and it does not need to be addressed.

Question: Section V,  Part 2.A, Additional Review Criteria:  The RFA refers to AHRQ policies on protections for human subjects, inclusion of both males and females and members of racial or ethnic minorities, or persons with AHRQ priority conditions.   In Part 2.D,Sharing Research Data (and in Section IV, the RFA calls for a plan for sharing research data.

Are these relevant considerations for Coordinating Center applications, given that the CC does not appear to have research, evaluation, or demonstration projects or subjects of its own, and presumably will not be doing projects that generate research data?  If it is, can the Government explain more fully what information is being requested in the CC applications with respect to these requirements? 

Answer:  Applicants for the CC should consider closely whether any aspects of your application might pertain to the protection of human subjects, even if such protection is limited to issues of confidentiality or privacy.   Please state in your proposal if you believe DHHS policies for human subjects protection do not apply in your case.  Similarly, if your application does not propose any sharing of research data, please state this.

These sections were included to assist Coordinating Center applicants that may choose to submit proposals where these requirements would in fact apply. 

All Research Center applications should have a section on human subjects protections, which is a DHHS regulation. They should state whether human subjects are involved and the protections involved.  If no human subjects are involved, then they should so state.  If data are public, applicants should state that there is an exemption (i.e., E4).  Even Coordinating Centers, which would manage human subjects indirectly, should address human subjects protections.

Similarly, all applications should address the inclusion of both genders, minors, and AHRQ priority populations.   The committee will decide whether the applicant has responded acceptably or unacceptably.  Relevant links in the FOA are:

Question: Section VI, Part 2: The support mechanism for the Coordinating Center is a U18 (cooperative agreement) and CERTs are funded via a grants mechanism, according to the RFA.  Can AHRQ clarify why Office of Management and Budget (OMB) clearance is needed for surveys that might entail asking identical questions of more than nine individuals? This is usually a requirement only of contracts.

Answer:  In the context of OMB clearance, cooperative agreements are currently treated the same as contracts. 

Eligibility and Cross-Funding Opportunity Announcement Questions

Question: There are several statements in the CERTs FOA about the "institution" rather than the "individual" being the applicant.."  What does this mean with regard to how I/my institution applies?  What sort of (and level of) institutional letter/commitment does the application require─e.g., Dean's Letter or Chancellor's Letter?

Answer:  All applicants should work with their institution's sponsored programs or research administration office however it may be named.  This will be the office that deals with obtaining outside funding for the institution.  Each institution should have an authorized institutional official who must endorse the application before it is submitted.  The applicant institution is legally responsible and accountable to the Federal Government for the use of funds provided and for the performance of the grant-supported project or activities.  The Principal Investigator, however, is responsible and accountable to the recipient organization for the proper conduct of the program.

As you probably noticed, the FOA mentions no requirement for a specific type of institutional support letter, but does mention institutional support in the reviewer's evaluation criteria, particularly in those sections with the headings Environment and Budget.   

Question: Questions about eligibility to apply have been raised with regard to the following instructions in the FOA: "Applications are encouraged from existing integrated programs, other than those already funded as CERTs (e.g., Centers funded through the National Center for Research Resources, General Clinical Research Center (GCRC) Program, the Centers for Disease Control and Prevention's Prevention Centers, AHRQ-funded Evidence-based Practice Centers [EPCs]).   Specifically, there was uncertainty whether the Centers listed in the parentheses could apply. 

Answer:  The existing integrated research programs listed in the parentheses are encouraged to apply, but not the four CERTs Research Centers that were awarded in April 2006 and which have continuing funding past 2007.  Those four Research Centers may apply for Health IT Research Center funding.

Question: Is there any prohibition against two CERTs applications being submitted from a single institution?

Answer:  There is no prohibition against two CERTs applications being submitted from a single institution.  Note that the FOA stipulates that an investigator may be a PI for only one CERT.  The CERTs FOAs indicate that funding decisions will consider program balance (avoidance of duplication and overlap in funding Research Centers, geographic distribution) along with other factors. 

Question: Can a CERT Center (either a Research Center or Coordinating Center) have two or more PIs?

Answer:  AHRQ does not recognize more than one PI per Center.  There can be co-investigators, but not co-PIs.  The applicant must designate only one person to be the responsible PI. 

Question: Would AHRQ consider it positive, negative, or neutral to have a single institution awarded two or more CERTs, such as a regular CERT and an IT CERT, a regular CERT and a Coordinating Center, or an IT CERT and a Coordinating Center? 

Answer:  AHRQ will consider application reviews, program balance, geographic distribution, and other factors listed in the FOA in making funding decisions.  

Question: How might it be possible for two or more institutions independently applying for CERTs funding to propose a joint project that leverages the resources of each institution? 

Answer:  If two or more institutions that are submitting separate applications wish to propose collaboration on a joint project, each proposal should have a description, plan, and budget for that project that would be sufficient unto itself, so that the application could be reviewed on its own merits and not in comparison to any other application.

Question: How much support (financial and intellectual) from the Coordinating Center should be anticipated by applicants for Research Centers?

Answer:  The FOA indicates that the CERTs Coordinating Center will handle monthly calls; Steering Committee agendas and logistics; project tracking; the production of CERTs program materials, including the annual report; cross-CERTs communication; and the dissemination of research findings.  The CC administers the public-private partnership committee described previously.  The extent of the "intellectual support" of the Research Centers by the Coordinating Center will be determined in part on the award of that cooperative agreement.   Each Research Center has its own AHRQ project officer to assist in overall direction. 

The Additional Guidance section of the FOA describes in bullet 15 what each Research Center must include in its budget funds for the Steering Committee, CC, and AHRQ activities.  That section begins "Each research center must include in its budget funds that allow for active participation in Steering Committee, CC, and AHRQ activities.."

Question: Are there already existing guidelines for specific details of how our center would interact with the Coordinating Center, or should we propose the details of this relationship in our application?

Answer:  Please refer to the Coordinating Center FOA, which lists 13 bulleted items listing the rights and responsibilities of the Coordinating Center under Section VI. Award Administration Information, Subsection 2.A.1.

Currently Funded CERTs

Initially funded in 2006:

Question: Can a PI at one of the four CERT Research Centers (RCs) funded beyond fiscal year (FY) 2006 be an investigator or subcontractor of a new RC? 

Answer:  In general, the FOA states this is allowed, but obviously specific circumstances of funding and project commitments need to be considered. 

With final year of funding beginning FY 2006:

Question: Please clarify the following language:  CERTs projects whose final year of funding begins in FY 2006 may submit new applications but may not submit competing continuations.  

Answer:  CERTs Centers whose final year of funding begins in FY 2006 can apply to this FOA with new applications.  If an institution wishes to include in its application a modified or expanded CERTs project where the final year of funding begins in FY 2006, any such project should be fully described in the application. 


Question: How can we submit separate budgets within the 424 application?  

Answer:  A recent announcement in the NIH Guide describes how this should be handled:

Question: Is the overhead for a subcontract with another group considered part of the applicant's direct cost in the budget?

Answer:  The indirect costs of a third party count as direct costs to the applicant organization.

Review issues

Question: How much familiarity will the special panel have with the CERTs program to assist in determining how much of the research plan should be taken up with background on the program? 

Answer:  The special emphasis panel will have information on the CERTs program, but not on the specifics of historically funded CERTs Research Centers or projects.  To avoid confusion or ambiguity, applicants should include all information that they consider necessary for a full evaluation of the scientific merits of their application.

Question: Will institutions that were previously CERTs be given any different consideration in review?  Did they have advance notice of this solicitation?

Answer:  All applications will be reviewed in the same fashion, assigned a priority score, and then funding considerations described in the FOA will be applied.  The existing CERTs had no advance knowledge of the FOA being released. 

Electronic Submissions

Question: What organizational structure and format should I use for the research plan?

Answer:  Please refer to the online guides for full information.  The research plan will need to be submitted with the relevant sections described in the FOA, which may be selected from among the following: Introduction to Application, Specific Aims, Background and Significance, Preliminary Studies/Progress Report, Research Design and Methods, Inclusion Enrollment Report, Progress Report Publication List, Human Subjects Sections (including four subsections), Vertebrate Animals, Select Agent Research, Consortium/Contractual Arrangements, Resource Sharing Plan,  and Letters of Support.

Return to Contents

Open Question Period for Conference Call Participants

In addition to the responses shown above to questions received in advance of the call, additional questions or points of clarification about the FOAs were asked on the call. 

Question: I'm still a little confused about how the number of projects spans over 4 years and your comment that basically the research plan should describe all research to be done over the entire 4 years.  My impression is that many of the CERTs do more than three to four or three to five projects over a 4-year period when you look at what's on the Web site.  So how do we handle that?

Answer:  The specific mention of three to five projects in this FOA is actually a new specification in this FOA.  AHRQ has not historically placed a limit on the number of projects.  Some of the projects that are listed for the CERT may in fact be leveraged projects that have some component of CERTs funding, as well as outside partner organizations.  Some projects outside the three to five that are proposed in this FOA may have been to be developed in collaboration with the AHRQ project officer via the U18 cooperative agreement mechanism.  Does that answer your question?

Question: What it implies then is that if we expect to do additional projects, then we need to allocate some part of our budget to having people and resources available to participate in additional projects.  Given the budget side, it seems three to five projects per side is a small number.

Answer:  The issue of the specific projects, their size, and how you fit them in with the infrastructure and other budgeting is actually going to be a matter for the individual institutions applying.  We cannot give you any more directive advice than that. 

Question: I have a variation of the last question; it's still not clear to me.  It seems to me that if the projects are pilot projects, in all likelihood, they would not last the duration of the 4 years of funding.

I had assumed, both by reading carefully the FOA and also by looking on the Web site of the existing CERTs, that these projects might last a year or two and that we would then continue to do other similar-sized projects.  That raises in my mind two problems that I'd like an explanation for:

1.      Within a 25-page application, it's hard to imagine giving enough information about both the infrastructure and, say, 10 or 12 projects that might be done over 4 years.

2.      The earlier comment that any project would have to be reviewed implies to me that perhaps if we develop new projects, say in Year 2, having set aside a certain component of the budget to cover new projects in subsequent years or in out years, they would then have to be submitted for review to AHRQ.

Could you clarify the issue of the kind, number, and length of projects?  Or is it expected that these projects that we've described would last the whole time?  And also the review of any new projects that might be developed?  If I could clarify that last issue, you said in your comments earlier and the FOA says that the infrastructure should be in place to sort of identify and develop new projects,; so that does imply that we would be developing projects over the course of the 4 years.

Answer:  In terms of describing the project adequately in 25 pages, we recognize the challenge with that page limitation.  We can only advise you to do your best in terms of allowing enough information for those projects to be reviewed.

In terms of future projects, one mechanism that we believe may be acceptable to the review panel would be to describe the process or criteria for an adequate and full review of the scientific merits of those projects.   Saying in your proposal, "We have allocated X dollars for future projects in this area," without stating how that funding decision would be handled, would not be well regarded, in contrast to saying, "We have a peer review process" or some process of consultation with AHRQ in making decisions about additional projects that might be included over the life of the funding.

Question: My question is about the themes and what are acceptable as themes.  Much of the research work in health information technology is cross-cutting, applicable to many different populations and different types of projects.  So how does that relate to the selection of the theme when examples of the theme have tended to focus on particular populations?

Answer:  Are you asking whether or not the Health IT Research Center should propose a theme other than health IT?

No, I'm asking about a proposed theme for a Health IT Research Center.  Can you propose a health IT theme that could be cross-cutting across many different populations, or do you have to propose a Health IT theme as you will apply it to a particular population, such as the example themes given?

Answer:  You can propose a Health IT theme that's cross-cutting.  You have to be careful not to duplicate the four existing themes that have currently been funded.

Question: The current document indicates that the start date could be July 15.  What should the current Centers do whose expiration date is much later, such as September 30?  Is there any special direction that they should use in looking at a potential start date?  Is there a preference that AHRQ would encourage those centers to consider?

Answer:  Any applicant, including the existing CERTs, can propose a desired start date in their application.  The expected start date would have to be no later than September 30, 2007.  The decision on the start date will be negotiated by AHRQ with the grantees in the light of funding decision dates, administrative issues, and other matters.  So the start date is something you can propose that will be handled in final negotiations, if applicable.

Question: Can we return to project number?  I thought that I understood what we should propose in this discussion but now I am quite confused.  The RFA mentioned three to five projects. It did not mention three to five projects per year, so I interpreted that to mean having to propose three to five projects for the entire 4-year period.  Subsequent discussion on this call seems to suggest the possibility of proposing additional projects, or more than five.  Could you clarify, please?

Answer:  In the section on additional guidance, the language says that funds should be budgeted to provide for staff support to undertake the design, development, and implementation of between three and five research and educational projects.  This will be a matter of review by the special emphasis panel.

One other way of looking at this is that we are looking at two to five projects overall. So you have flexibility in how broadly you plan and define those projects.  It could be one project for the first year, then moving on to the second project in the second year, or you have the flexibility of proposing two to five projects that would each require the entire 4 years of research.  Of course, there will be the judgment of the peer reviewers, but this gives flexibility in how you can propose a project.  We don't want to constrain it to making it only 1 year for each project or 4 years for one project.  That actually is up to you.

Question: We interpreted the RFA to suggest that the project could be relatively broad in scope with different components and different parts as long as we could make a clear justification for the interrelatedness of the component parts to the special emphasis panel.   Would that be a fair statement on our part?

Answer:  Yes.  We think that's actually a good interpretation in terms of having a broadly based project that might in fact result in a variety of research products, such as several publications or other matters.  This perhaps gives you a little better sense of the breadth to which we may be applying the term "project," so that you might think of them more as major project areas.

Question: To follow on with one additional question, we had also been presuming that it would be effective to weave an educational component into each of the projects, as opposed to setting one of the projects aside to be mainly educational.  Is that also a fair interpretation?

Answer:  That is going to be a matter for the review panel to decide.

Question: When you write these three to five proposals, are you held to doing all three of those five proposals or can they can be modified or substituted once you get into the system?

Answer:  We don't encourage individuals to propose projects that they wouldn't plan to carry through.  When the programs are up for continuation, modification, or substitution on an annual basis, this can be a matter of discussion and renegotiation with the AHRQ project officer.  There is flexibility over the life of the project for modification based upon programmatic input.

Question: Can the letter of intent be sent by e-mail?

Answer:  Yes.  It goes to Amy Lindinha.

Question: This question has to do with the outcome specified for each of the research or education projects--the measurable outcome.  We were wondering whether that has to do with outcomes in the study population or measurable outcomes in improving therapeutics in the United States.

Answer:  It is difficult to answer that question in the specific.  Overarching goals of the CERTs program are that the research and educational projects undertake and have some measurable benefit, ideally in influencing health outcomes.  We will leave it to the individual applicant and the review panel to comment on the specific merits of what's proposed.

Question: Are we supposed to have separate human subject details for each of the pilot projects in terms of numbers of children, elderly, minorities, etc.?

Answer:  For purposes of your application, it's probably most important to make sure that the reviewers are clear that human subject protection, if it is relevant, has been addressed.  It really is the applicant's prerogative in terms of how you handle the material.  You could have one major human subject section for the entire application or, at the end of each project; you can have human subjects information It's really your prerogative.

Question: Would it be okay to have a blanket statement that covers all the projects saying something like, "Human subject protection will be addressed by the appropriate institutional review board," without going through all the details that we would for an RO1 or some other type of grant in terms of listing targeted and planned enrollment, talking about inclusion of women and minorities, and similar levels of detail.

Answer:  You have to provide some sort of detail because the reviewers will look at your application to see whether you really do have a plan to ensure the protection of subjects in your specific projects.  They want to see that protection for human subjects is ensured.  A broad statement that says, "We'll take care of this," will not necessarily provide sufficient reassurance to the reviewers.  You'll have to balance the level of detail you provide with what you think would assure the review panel that human subject concerns are satisfactorily addressed.

Question: Regarding the Coordinating Center, under special eligibility criteria, special requirements, the second bullet, it notes that the CC convenes the Steering Committee which meets three to four times per year.  Travel will be coordinated through the CC. Does that mean that the CC would have a role in assisting the sites and other participants or is it simply coordinating travel for its own staff?

Additionally, it notes that the meeting will be held at the Eisenberg Center, at the AHRQ Conference Center. Is there a charge for use of that facility that the Coordinating Center needs to budget?

Answer:  There is no charge for the Eisenberg Center's facilities.

Question: Then going down to bullet number nine, it said the CC application must have a business plan.  Could you define that a little more in terms of whether this should be something that entails the sub-budgets or describes specific functions? 

Under bullet 10, it says the CC will develop programmatic products for the CERT.  I wonder if you could get us some guidance about the number and complexity of products?  Also, who should be responsible for budgeting any sort of reproduction and distribution cost?  Related to that, under qualification, bullet number five, it said the CC must be able to provide writing and editorial assistance on an ongoing basis for meeting minutes, which is clear, but then it also refers to "written manuscripts and research summaries" and says that the CC may be asked to develop original manuscripts.  This area could be very limited or very open ended in terms of the level of support you would be expecting the Coordinating Center to provide to the individual sites, looking at the number of manuscripts they could be expected to produce.  Could you say a little more about the level of detail we should use in budgeting that?

Answer:  We can answer some of your questions.  We will defer the one about the business plan to subsequent e-mail communications.  On the issue of travel coordination, the work of the Coordinating Center is largely in organizing.  Other than Coordinating Center staff who might need to travel, the actual expense of travel is borne by the Research Center.

In general, the Coordinating Center would not bear the printing cost of any materials that are produced.  The Agency bears those costs.

(Note: Follow-up of the unanswered questions from this caller was still pending at the time this Summary was prepared. We are posting the Summary without this information to avoid undue delays for those who were unable to participate in the call.)

Question: To be clear on the plan for sharing research data in this FOA, is a data-sharing plan required?

Answer:  The FOA refers to directions should you plan to be sharing data.  That is certainly the prerogative of the applicant to propose.  If it is proposed, the necessary information should be provided.

Question: There was discussion earlier about providing a bibliography for the PI.  Is that an additional requirement to the biosketch, which would have those publications listed?   

Answer:  The question that we answered previously pertained to the PI of the Coordinating Center.  Some may have questioned whether that PI would have to cite any publications or bibliography.

Question: Our team conceives of this project as four specific projects plus one that's in underlying infrastructure support.  Do I understand that we would have 25 pages describe each of the four projects as well as describe our infrastructure needs?  Or do we have to combine all of that into one and relate that to the infrastructure?  Should we conceptualize the infrastructure as something separate or something that has to be built into our projects?

Answer:  For your four specific projects and the underlying infrastructure project, whether you should develop them separately or build the infrastructure into the projects is really a matter of your choice in your application. The page limitation for your entire research plan is 25 pages.

Question: How would we treat intellectual property regarding the software that we develop as part of this project?

Answer:  For software that is developed under a grant mechanism, the copyright or the intellectual property right remains with the grantee.  This is the way that it's been treated in the past in the Federal Government.  However, as with many other products that are created under a grant mechanism, the grantee agency─--the Federal Government─--retains the right to distribute and use that product free of charge as we choose.  So the property, --the intellectual property rights, --would remain with the grantee for software domain, but the Government has a license to use and distribute the product free of charge.

Question: Regarding the budget justification, I noted there's only one attachment allowed.  How did you want to structure the budget justification?  Did you want to structure it for all your projects separately or combined  for the entire proposal?

Answer:  Applicants are allowed to structure their budget justification as they think fit for their proposal. The budget justification will have to clearly identify which project and organizations the budget represents.  A project-by-project format usually works well when presenting the budget justification for a multiple-project application.  It must be clear to the reviewers and to our staff what costs are associated with each project.  The budget justification also needs to address costs for future years and pay particular attention to explaining any fluctuations in costs in future years.

Question: Regarding infrastructure and the predefined projects, it's difficult for me to see how you budget for infrastructure when you have a budget and budgets for three projects that would have to add up to the total.  I'm a little confused about project-specific budgets and how actually one would include infrastructure and allocate infrastructure to a specific project. Can you clarify this for me?

Answer:  Traditionally, we have seen separate infrastructure budgets and then individual project budgets. If for some reason you feel that for your proposal you can allocate the infrastructure under the projects, which would be fine.  But a centralized infrastructure project is acceptable.

Question: Was that mentioned in the RFA?

Answer:  I don't think that the level is detailed.  I think it just said that you need to have an infrastructure component.

Question: So a separate budget for infrastructure is appropriate then?

Answer:  Yes, that's fine.

Question: I'm calling in regard to budget questions.  Since this is an electronic submission,  and I must use the application budget form to capture multiple-budget projects, am I just to fill out the composite budget on the main form and then attach the various spreadsheets that break down that composite budget by the projects?    If so, where is that attachment going to go?

Answer:  That's an excellent question and, hopefully it is answered in the Notice that was just issued on November 7.  If you refer to and read that and have additional questions, you can contact Michelle Burr directly. But we do give detail on how the budget will have to be done separately, using separate spreadsheets that you will devise yourself.  They will have to be converted to PDF format and uploaded as part of the budget justification.

Question: I did get the addendum for the budget submission instructions.  However, it's not exactly specific as to where the attachment should go on the actual electronic application.  It does tell me how to name that stuff, which is fine.

Answer:  It does state that it should be submitted as part of the budget justification section of the application.

Question: It was stated that the applicant should be structured with the flexibility to develop new projects on an annual basis. Do you have guidance as to how much funding should be budgeted for this?

Answer:  We can't give you any specific guidance as to the amount of money that you might want or the mechanism you would set up to fill that flexibility.  But as we described previously, on an annual basis and in consultation with AHRQ as to its programmatic needs, you might specify a process that would provide some assurance to the review panel that whatever projects were undertaken would have high scientific merit.

Question: Since the applications will not be reviewed until March 2007, is there any possibility of extending the deadline for submission?

Answer:  We cannot extend the deadline for submission.  There are other processes built in between receipt of the application and the meeting of the review panel.

Question: I have a question about the budget and the Coordinating Center.  Does that amount include direct as well as indirect costs?

Answer:  Yes, that amount includes direct and indirect costs.

Question: Regarding the 25-page limitation, I wanted to make sure that budget justification and appendices are separate from those 25 pages.

Answer:  That's correct.

Question: I have another question in regard to the budget, and how it is related to the sub-award budget. I'm not sure exactly how much detail you need. I notice, for example, that the University of Washington requires different, more detailed information than the system will let me see uploaded.  So can you give some direction on what we should ask on some of our sub-awards in terms of the documents that they will need to submit?

Answer:  For any projects that involve sub-awards, the sub-award budget needs to be the same detailed budget that you will submit for the main project.  It's the same level of detail.

Question: I have one attachment also that's allowed in this, the budget justification.  Do you require details in terms of materials like resources and environment?

Answer:  Are you talking about the resources and environment for the overall project or for each? 

Question: My question is about the sub-award.

Answer:  For the sub-award, I believe that should actually be included when you describe the resources available to the overall project.  I think you can address each organization separately.  You can e-mail Michelle Burr if you have further questions and we'll look into it.

Return to Contents


Since there are no further questions, we can conclude our call.  We will be sending information to those of you who participated about when and where the transcript of this call will be available as well as the slides.  Thank you all for your participation and for your interest in this funding opportunity.

Return to Contents

Current as of November 2006

The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care