Technical Assistance Call for Applications to AHRQ's Health IT Announc
This document summarizes the technical assistance call for AHRQ-sponsored Health IT funding opportunity announcements (FOAs). Select to access the FOAs.
The technical assistance teleconference was held at the Agency for Healthcare Research and Quality (AHRQ) conference center in Rockville, MD, on January 13, 2009. If after reading this document you have any questions or comments, contact healthITgrants@ahrq.hhs.gov.
Rebecca Roper: Good afternoon. As identified, I am Rebecca Roper and I'm one of the members of the Health IT (Information Technology) Team at the Agency for Healthcare Research Quality (AHRQ). We are very pleased with the number of participants this afternoon, both around the table with me at AHRQ and across the Nation.
There are many people who are excited about these Health IT Funding Opportunity Announcements (FOAs), and we hope that translates into many very competitive and meritorious applications coming in to AHRQ. So before I get started, we'll take an opportunity to let you know who the people are at AHRQ who are here to support you through this process and who are here in person. Starting with Boris Aponte, introduce yourself and maybe tell us where you work in AHRQ.
Boris Aponte:Good afternoon, everyone. My name is Boris Aponte, and I'm a Scientific Review Officer (SRO) in charge of the study section for health care information technology.
Ali Azadegan:This is Ali Azadegan, another member of the Division of Scientific Review (DSR). I am a Scientific Review Officer, who from time to time, [conducts extramural peer review] of the training applications that come through, some of those [applications] have to do with health IT.
Jon White: Hi, my name is Jon White and I direct the Health IT portfolio at AHRQ.
Ileana Ponce-González: Hi, my name is Ileana Ponce-González, and I am a Scientific Review Officer.
Erin Grace: Hi, my name is Erin Grace. I'm a member of the Health IT team.
Teresa Zayas Cabán: My name is Teresa Zayas Cabán; I'm also a member of the Health IT team.
Matthew Quinn: This is Matthew Quinn; I'm also a member of the Health IT team.
Debbie Rothstein: I'm Debbie Rothstein, Advisor for the Office of Extramural Research, Education and Priority Populations (OEREP) at AHRQ.
Michael Hagan:[I am] Michael Hagan, Economist and Program Officer and contact on research priorities around value.
Rebecca Trocki:[I am] Rebecca Trocki, Program Analyst, Division of Scientific Review (DSR). I work with Ali Azadegan and Boris Ponte.
Steve Bernstein: [I am] Steve Bernstein, member of the Health IT team.
Michelle Burr: [I am] Michelle Burr with Grants Management.
Kay Anderson: [I am] Kay Anderson, the program officer for career awards grants a.k.a., K-awards) grant.
Bob Mayes: [I am] Bob Mayes, Health IT.
Rebecca Roper: Thank you. Collectively, we will be providing an overview of the three Health Information Technology (IT) Funding Opportunity Announcements (FOAs) that were published last fall (September 2008). I just need to clarify that the information provided in the actual funding opportunity announcements is the information that should be used in the sense of direction of how applicants respond to these FOAs. We hope to rearticulate or restate in a different way the expectations articulated in those FOAs. Should I trip myself up and make a misstatement in this oral presentation [that contradicts guidance in the health IT FOAs], disregard [the oral statement], and use narrative of the Health IT FOAs as your guidance.
Does anyone care to add to that?
Jon White: Mention the transcript; that there'll be a transcript.
Rebecca Roper: Oh yes, indeed. We will have a transcript that will be created based on our presentation and your thoughtful questions and answers, both those that have been posed previously and we'll respond to at the end of the call, and any new questions that you will articulate and we'll respond to today. We're hoping that that transcript will be posted on our Web site [https://www.ahrq.gov/fund/tacall.htm] by a week from Friday, by January the 23.
As I stated, our intent is to highlight the Health IT FOAs and maybe provide some clarification on issues. Historically, AHRQ Health IT grants initiatives have really focused on one-time Request for Applications (RFAs) and some of you may be familiar with the Ambulatory Safety and Quality (ASQ) Program and the RFAs that were published as part of that Program back in 2007 and 2008 [HS-07-007; HS-07-006; HS-07-002; HS-08-002] and hopefully several of those folks who were applicants to those RFAs are interested applicants who are joining us today.
In the fall , AHRQ chose to publish three FOAs using different types of funding mechanisms:
- The small research grant to improve healthcare quality through health IT, that's an R03 [PAR-08-268].
- An exploratory and developmental grant to improve healthcare quality through health IT, an R21 [PAR-08-269], which is a little bigger grant as far as total cost and a little more robust research project is expected.
- The R18 [PAR-08-270], utilizing health IT to improve healthcare quality, which may be a more familiar mechanism based on your experience at AHRQ with respect to applying to R18 grant initiatives.
So we're now on the Slide Number 5 if you're looking at your PowerPoint® presentation [https://www.ahrq.gov/fund/tacall.htm]. Slide 5 provides a tabular—a table presentation—of the overview of the AHRQ-sponsored health IT FOAs.
For the purposes of this call, those are the three funding opportunity announcements we are going to focus on. As you are aware, AHRQ has recently published two new versions of its program announcements for the R01 [PAR-09-070] and the R18 [PAR-09-069], and a little later we will address how those are different [from the three health IT-oriented FOAs that we will be discussing] today with respect to health IT.
You also are aware that in the fall  we published a Special Emphasis Notice [NOT-HS-08-014] to encourage applicants who have a health IT focus for either career development or research dissertation grants to submit those applications to AHRQ and we'll talk about them just briefly. We have Dr. Kay Anderson from that group who will be able to provide responses to any questions that you pose in the open forum. But for today, let's take about 5 minutes or so and look at Slide Number 5 [found at https://www.ahrq.gov/fund/tacall.htm]. It synopsizes some of the milestones related to the three funding opportunity announcements. To put the FOAs in context, we can highlight their scopes. So for the R03, three different types of research grants can be supported through our R03:
- Small pilot and feasibility or self-contained health IT research projects.
- Secondary data analysis of health IT research.
- Economic prospective or retrospective analysis.
The R21 supports conduct of short-term preparatory pilot or feasibility studies that are needed to inform future health IT implementation, which may include, but is not limited to, the conduct of a health IT research grant. And what I'm reading comes right from the FOAs but our presentation gives you a little context.
The R18 supports demonstration research grants that rigorously study health IT implementation and use to improve the quality, safety, effectiveness, and efficiency of health care in these settings. So if you look at the authorized duration, all three mechanisms are currently authorized for 3 years.
The earliest submission date for the R03 and R21 is coming up; it's February 16, in about a month. The recurring due dates are also specified. But notice for the R18, that first due date is right around the corner; it is on Sunday, January 25.
Whenever a recurring due date [for submission of an eGOV application] falls on either a weekend or a Federal holiday, as stated on the FOAs, you have until the next business day at 5 p.m., local time for the applying institution, to submit your application. Due to challenges in e-GOV [process] and given that any type of hiccup can occur, we strongly encourage you to submit your applications before the recurring due dates as posted.
With respect to the differences across the different health IT FOAs, you'll notice for both the R03 and the R21 that one may request a maximum project period of 2 years. If your project design is such that you can accomplish what you need to in under 2 years, you are welcome to specify a shorter duration. Similarly for the R18—it's a 3-year maximum project period [although a shorter duration may be requested].
For those of you who are familiar with grants, there are no-cost extensions that may be requested [toward the end of the authorized grant period] but that's a different issue [known as a] no-cost extension, it would be in addition to the maximum project period for which you would be articulating a research plan and a timeline in an application process.
If you look at the total cost over the grant lifetime—total cost is comprised of both direct and indirect costs; for the R03 the total cost is $100,000, for the R21 it's $300,000, and for the R18 it's $1.2 million. Within a given year, the total cost does have a maximum value for each funding opportunity announcement.
You'll also notice a progression with respect to Part D, the research plan. For the R03, we allow a 15-page limitation; for the R21 it's a 20-page limitation; and for R18 it's a 25-page limitation. The number of pages allotted really reflects the anticipated complexity of the research projects and respective analytical plans that would be posed. So we try to give you ample room to articulate well your research plan.
If you don't get funded the first time around, I encourage you to consider coming back [with a resubmission] based on your areas of interest for research and the nature of the comments that you receive; you may submit one resubmission. [For the R03, you have one resubmission opportunity.] You have two more bites of the apple, so to speak, for the R21 and the R18; and whether or not you proceed with the resubmission, that's completely up to you.
Also, for each funding opportunity announcement you have a requirement to have an introduction to describe your resubmission: [the length of that introduction] is one [R03], two [R21] and three [R18] pages for the progressively more complex health IT funding opportunity announcements.
With that, I'll see if anybody around the table seeks to provide clarification.
I want to emphasize to you that unlike the one-time RFAs, you have an opportunity for resubmission [with these health IT FOAs]. So if you don't succeed [get funded] the first time and you think you have a good idea, come back to us [Resubmit].Really, this is what we're hoping will happen; we'll learn by your application process and you'll learn by our response and the input of the extramural peer reviewers as well.
Debbie Rothstein: One comment I would make is if anyone is thinking about trying to get funding in Fiscal Year (FY) 2009 for AHRQ, this upcoming grant cycle [due dates in January, February, and March], this is the only cycle that will be considered for FY 09 funding; all future cycles based on how long it takes to receive, review, and make funding decisions, they'll all be bumped into 2010 and beyond.
Rebecca Roper: Moving on to our special emphasis notice (SEN) for health IT focus on career awards. There are three different career awards that were featured in the health IT focus. And just so you know, the (K) awards, the first for this cycle, are due February 12.
The grants for the Health Services Research (HSR) dissertation R36 for the first cycle, they're due February 16. In the body of the PowerPoint® presentation, you have the contact information for (Kay Anderson) who is in charge of the (K) awards and (Brenda Harding) who is in charge of the dissertation research grants.
Any additional input here? Okay.
Now, we're going to take a few minutes to talk about those issues that are common across all three health IT funding opportunity announcements. And after we give those presentations, we'll begin to talk about what differentiates each funding opportunity announcement.
Examples of health IT were articulated in each of the FOAs. But, they are examples; there are other types of health IT that can be quite applicable to the FOAs; for example, bar coded medication administration (BCMA) is another type of health IT that can be appropriately applied in this particular series of health IT FOAs. There are telemedicine applications that can be quite appropriate. To the extent that you can demonstrate the appropriateness of your health IT, you're encouraged to submit an application.
We have three research areas with respect to health IT. You'll notice these research areas are consistent across the health IT-oriented FOAs that we're talking about today. These three research areas are also echoed in what I will call the AHRQ-wide FOAs-the R01 [PA-09-070] and R18 [PAR-0-071] that were just issued.
So applications [to AHRQ] that involve health IT must identify themselves as addressing at least one of the three research areas:
- Medication management (described in the synoptic fashion).
- Patient-centered care including coordination of care and use of electronic exchange of information.
- Health care decisionmaking through the use of integrated data and knowledge management.
Now the breadth of how these particular concepts [research areas] is quite large and the examples given are quite varied. We do not expect a given application to address all the different forms, for example, of healthcare decisionmaking in a given application. Also, we recognize that some applications may address one or more of the research areas and that's fine to articulate.But [you should] also articulate to us which particular one [research area] is your primary area of focus.
An important area for reinforcement in these health IT FOAs is the appropriateness of the research settings that you've identified. For the purposes of these health IT funding opportunity announcements, we require that the health IT being implemented or the health IT evaluation that's going to be conducted [be related to health IT implementation that] has occurred in ambulatory care settings [or in transitions in care settings], and we have listed several example ambulatory care settings.
You may have other ambulatory care settings that are quite applicable. You need to just demonstrate the appropriateness of them in your application. One thing to point out for these health IT FOA applications is that those [proposals which would] focus on health IT implementation in emergency departments or the transition from emergency departments to an inpatient hospital setting would not be permissible and would not be reviewed.
So again to reemphasize that, within each of the health IT FOAs, we specify some nonresponsive research settings where an application that features health IT implementation in non-ambulatory settings such as a hospital, an emergency department, a skilled nursing facility, or an inpatient mental health facility for purposes other than facilitating transitions in care to and from an ambulatory setting would be considered nonresponsive to these FOAs and will not be reviewed.
We'll have some examples later on [in the question and answer section of this presentation] of what we consider to be appropriate and some nonappropriate health IT care settings for these health IT FOAs.
(The following statement from later on in the call was repeated for emphasis: We anticipate that there will be many different variants of allowable ambulatory settings and/or transitions in care and have not chosen to select an exhaustive or present an exhaustive list; we just encourage you to articulate and demonstrate why the care provided [in your proposed health care setting] is commensurate [with the level of care provided] in an ambulatory setting.)
Several examples within questions and answers are provided. Go to the PowerPoint® Presentation that accompanies this Technical Assistance Call, slides 10, 11, 41 (Mobile Van and Ambulance examples), 42, and 43 (allowable overnight care settings).
Rebecca Roper: In each of the funding opportunity announcements there is a section IV. 6, which identifies specifics [required for a given] proposal. First, let's talk about the requirements that are consistent across the health IT FOAs. You need to provide a detailed description of the health IT intervention.
We encourage you to describe the interface with preexisting health IT, if any. And, if possible, the health IT intervention should conform to Federal and other interoperability standards and use certified products. If this isn't feasible just give us an explanation.
Also, depending on the type of research you're proposing, for example, if you're conducting a secondary data analysis or if you're conducting an economic analysis you may not actually be implementing a health IT,you [should] discuss the type of health IT that was implemented about which you're conducting your analysis.
Here's another issue that pops up regularly: guidance on software/hardware and/or equipment purchases. In combination, over the lifetime of the grant we expect that such purchases will be 20 percent or less of the total cost. But in your circumstance, you may require more money; if you do, we require that you provide a justification. You will be evaluated on the appropriateness of these purchases.
Next is something that you will see in the health IT FOAs that has a new dimension to it and it has to do with the research methods section: identify if a privacy and security plan is necessary.For many of you who participated in the earlier AHRQ health IT requests for applications for the Ambulatory Safety and Quality (ASQ) Program [you will note] that a similar requirement was put forth. But we have tried to provide you more guidance with respect to what we anticipate and what we need to see.
To reinforce the importance of the privacy and security plan and not to discourage you from taking the appropriate amount of time and narrative to address it, we ask that you provide a concise summary of the plan in your Part D [PHS 398 Research Plan].In your page limitation for research methods [for each health IT FOA], you provide a concise presentation of your privacy and security plan.
But in Appendix A, if you are engaged in health IT implementation, you have an unlimited opportunity to articulate the privacy and security plan as you've identified it at the time of preparing your application.
All [submissions to these FOAs must be made electronically.] As you know in December 2008, our three health IT FOAs were updated to include the Adobe forms for the new process. So if you haven't looked at the electronic version of the health IT FOAs in the last month, you may not have seen that the forms are available. I strongly encourage you to download either Adobe® Reader® 8.1.3 or 9 (it's available through the link that you have in the health IT FOAs) and to download the [application] forms and take a look at them, if you have not engaged in this process before.
I also want to reemphasize that for AHRQ, we require that you use a Research and Related budget form [detailed budget]. So if you should accidentally use the (PHS 398) Modular Budget Forms, your application will not be reviewed. So be very careful.
Debbie Rothstein: One other side note to that, I'm assuming most of you have seen this but if you have not previously done an electronic submission you will note that the funding announcements indicate that you and your institution have to actually register in two different places: eGov: [http://www.grants.gov/applicants/get_registered.jsp] and eRA Commons [http://era.nih.gov/ElectronicReceipt/preparing.htm]. You should plan that it could take as long as 2 weeks, so that you should also begin thinking in terms of what you need to do to prepare for electronic submission almost immediately.
Rebecca Roper: Thank you. Also identified within each FOA are eligible institutions. And I suggest you look at the narrative in detail. But to highlight: For-profit organizations are not eligible to receive funding as the awardee institution for an application. And foreign institutions are not eligible to apply as the primary awardee institution although they can be part of a consortium and we'll get to that in a little more detail later.
All right, principal investigator (PI), you'll notice in the announcements that the terms principal investigator, PI, and PD, project director, are used interchangeably. PIs may come from a variety of different background areas. The extent to which a PI is appropriate is really in terms of the particular funding announcement to which you are responding and the requirements identified [within that FOA] and the research scope of the particular application.
One thing that's standard across our three health IT FOAs is that a PI should devote a considerable portion of time to the project proposed. If less than 20 percent time will be devoted, the application must include an explicit justification. Applications should describe PI responsibilities and backgrounds. The level of experience expected for our PIs really varies across the type of health IT research project.
We are hoping through these announcements not only to generate particular types of research topics but also to cultivate health IT research capacity among interested professionals.
I'll just see if anybody has any general comments. Now we're going to forge on to the FOA specific issues. We'll start off with a small research grant to improve health care quality through health IT [R03] and then we'll go up the line to the R21 and R18.
So the R03 really supports a wide variety of research designs. It's foreseeable that you may have unique ideas and you could [submit] different R03 proposals at the same time. But let's just talk about the three basic types of research design [for the health IT R03].
There's the small pilot and feasibility or self-contained health IT research project. There's the secondary data analysis of health IT research. And there's the economic analysis of health IT implementation and that might either be prospective or retrospective. So we hope we've given you lots of choices here.
In general, for the R03, you need to pay particular attention to the research criteria given the type of study you propose. So for the small pilot and feasibility or self-contained health IT project we expect it to inform future development of health IT strategies for health IT research or implementation activity. This is preparatory work.
The PI is not expected to be extensively published or have prior leadership in the research project. And consistently across all types of research design and the different mechanisms, applicants are encouraged to consider a wide range of scientific methods including both quantitative and qualitative methods. The study design and analytical plan should be well justified and consistent with specific aims. But we anticipate that they will be quite different in their approaches.
The second type of design in R03 is secondary data analysis of health IT research. And this is intended to generate insight regarding facilitators and barriers to health IT implementation through evaluation of factors impacting utilization and implementation.
So some of the recipients [of the earlier] Ambulatory Safety and Quality (ASQ) RFAs may be in a good position to put forth new [secondary data] analyses—although [R03 secondary data analyses] proposals are not limited to those grantees. Secondary data analysis is not a regurgitation of previously conducted analysis but new analysis, evaluating various characteristics, workflow, and integration with pre-existing health IT and to test hypotheses or synthesize data derived elsewhere.
There are additional requirements that I encourage you to carefully consider when you're putting forth your proposal with respect to secondary data analysis. Again, I want to emphasize that [secondary data] analysis may be related to but must be distinct from the specific aim for the original data collection and analytical plan.
Also keep in mind that the extramural peer reviewers will be assessing the merit of your application and will be looking at what we have articulated as being important in the research methods section and that is reflected in the review criteria, which are unique when it comes to approach for each of the types of R03 study designs that you may pursue.
We've got something new: the economic analysis of health IT implementation.It is our third broad category of type of R03 research that can be conducted. It must focus on a specific health IT implementation project. That project could either have been sponsored by AHRQ or others. Here you have two options, which may provide you opportunities that may be a little confusing.
[First] we have a concurrent prospective economic analysis. So let's say for example you are putting forth [submitting] a health IT R18 implementation study [PAR-08-270] and you would also like to prospectively conduct an economic analysis of that implementation [during the same due date cycle]. You may put forth both the R03 for the economic analysis and the R18 to evaluate the health IT implementation and use project at the same time.
You'll see within the R03 and the R18, they identify how you cross reference both of those studies—so that we can make sure that they're linked up when the decisions are made with respect to funding.
We also give the opportunity for retrospective economic analysis. Here it's important that you identify a predecessor health IT implementation project and make sure you describe the conditions for use of the data. As with regular secondary data analysis, the extent to which you articulate clearly how this R03 is distinct from your previous work will be very important.
As we highlight, in your design to conduct sound economic evaluations of health IT implementation and use, you should consider to whom the benefits accrue, include evaluation of financial and nonfinancial costs and benefits of the companion health IT implementation project.
In the body of the R03 FOA, we highlight more information with respect to what can be emphasized in the economic analysis.
Michael Hagan: I was just going to add that the interpretation of benefits and costs is broad, that within the organization one could consider the distribution of costs and not just costs in terms of what's paid for but the cost of diverting resources to the implementation of IT or input of information or for maintenance or whatever so that the cost is considered in a broad way.
Rebecca Roper: Thank you, Mike.
Rebecca Roper: As we've discussed, there are specific R03 submission requirements which we've pretty much highlighted so far. We'll go through the submission requirements. There are several different categories [of the types of submission requirements]. Again, I really encourage you to look at the instructions in the health IT FOA and make sure that you're meeting them and can provide as much information to demonstrate that you've addressed these different issues.
It just so happens on Slide 26 of the PowerPoint® presentation, budget was inadvertently dropped off, but that is one of the other requirements that you need to address.
As we already articulated, there are specific review criteria depending on the type of R03 to be evaluated. Now, unless there are anymore comments about the R03, we'll go on the R21, exploratory and developmental grant to improve health care quality through health IT.
This [R21] mechanism offers $300,000 total cost over 2 years. So like the R03, you can take up to 2 years [to complete], but the R03 has a cap of $100,000 total cost. The R21 is to conduct a short-term preparatory pilot or feasibility studies that are needed to inform future health IT implementation, which may include but is not limited to the conduct of health IT research demonstration grants.
The R21 grants are more comprehensive and broader in scope than the small self-contained health IT research projects. So a question sometimes is raised [by a would-be applicant], I don't know which is more appropriate, should I come in under the R03 or the R21? As you look at the specificity of the two FOAs, you can choose [the one that best meets your research objectives] but sometimes just keeping in mind the actual total cost limit, may inform your best route to pursue. So if your small self-contained health IT research project requires more than $100,000 total cost, the R21 would be the better way to go.
Also, to emphasize again for the R21 personnel, the PI is not expected or required to be extensively published or have extensive experience in leadership as is traditionally expected for R18 and R01. This [information] is within the body of the health IT FOA document to which you will be applying. This [information] will also be [considered by] the extramural peer reviewers who use the same information to evaluate the merit of your application; they will be taking that into consideration also.
Unlike the R03, though, you have two bites of the R21 apple. So if it doesn't go well the first time around and you're interested you can come again; you have two more times to get funded. You have an additional allotment of two pages to address how your revised application responded to previous concerns. Again, we anticipate a wide range of methods, both qualitative and quantitative, to be appropriate to the analytical analyses that are proposed [in the R21] applications.
Now we're in the more familiar territory of the R18, Utilizing Health IT to Improve Healthcare Quality. This [FOA mechanism] supports demonstration research grants that rigorously study health IT information and use to improve the quality, safety, effectiveness, and efficiency of health care and ambulatory settings and transitions between care settings.
It offers $1.2 million over a 3-year period. The research design and methods section has other requirements that provide more specificity compared to the R03 and R21; more rigorous research evaluation is expected. [For example,] you'll see that we have guidance with respect to clinical and staff perspectives being solicited on the utility of health IT as it is evaluated or justification for not including them.
Also, we expect that patient or caregiver's use of health IT should be considered and a variety of other issues that we ask you to address, including unintended consequences. The review criteria are quite specific for this mechanism and really take the time to look at those as you form your proposal because those will be the review criteria that the extramural peer reviewers will be using to assess the merit of your application.
Anything else from the group? Okay, so we received a series of question and answers from our interested public, and we'll just take a few moments to go through the questions posed and our responses. And then we'll take a [short] break and should there be any outstanding questions we'll work with the operator to solicit your input and then we will provide feedback as we can.
So here we are on Slide 37, "Can there be a co-PI?"
No. AHRQ requires that the lead institution designate one and only one individual as the project's principal investigator (PI). This does serve as a nice segue.You will notice that NIH [the National Institutes of Health] is considering different types of review requirements and application requirements that may be considered by AHRQ.AHRQ will be making decisions whether or not we will subscribe to those changes or have our own policy. Is there anything else Debbie Rothstein or the Office of Extramural Research, Education, and Priority Populations (OEREP) cares to add?
Debbie Rothstein: No. Other than what we'll be doing is everything that's articulated in these FOAs and should there be any changes, there will be notices about that. But, no additional information is provided at this time.