You Are Here:
Please go to www.ahrq.gov for current information.
The document below is an amended transcript of a Web chat that took place on September 20, 2001, among the AHRQ staff and potential applicants to the CAHPS® II RFA. Questions have been numbered for reference. In some cases, the response has been edited for clarity or expanded to include additional information that may be helpful to applicants. Even those who participated in the Web chat may find it helpful to review these responses. Applicants continue to E-mail us with questions, so we will continue to post these new questions and their responses as they are received.
Moderator: Welcome to the Web Chat. Dr. Crofton, do you have any opening comments?
CAHPS: Yes, I'd like to say hello to everybody and thanks for joining us. With me today is Chuck Darby, and Al Deal from our Grants office. We also have two new members of the CAHPS® Team—Kathy Crosson and Carla Zema.
1. I have tried to access the TalkingQuality Web site (www.TalkingQuality.gov) that was referenced in the RFA, but the site is password-protected. How can I access this?
Unfortunately, the TalkingQuality Web site is not available at this time. The URL takes you to the beta site that was tested earlier this year. If you are interested in seeing the beta site, please contact Maria Wey at MWey@ahrq.gov for access information.
2. Will grantees have automatic access to NCBD (National CAHPS® Benchmarking Database) data and reports for analytic and testing purposes? For example, will the grantees be able to test the NCBD reports with providers?
Regarding the NCBD data, there are established rules and procedures for access to this data set. Ready access is provided to the NCBD data as long as those wanting access follow a standard set of procedures for maintaining confidentiality of the data. Regarding testing NCBD reports with providers, this is a possibility. Once we assemble the grantee team, we will decide together which reporting projects will take priority. This will be done through the steering committee.
3. What are the minimum details required for the letter of intent?
Organization and the name of the Principal Investigator and their alternate. If you feel like providing us with more information about team members, that would be helpful, but not necessary. More information is on page 22 of the RFA.
4. Page 8 of the RFA discusses convening experts in a variety of areas, including "information" design. Could you elaborate the types of skills entailed in the area of "information" design?
Information design refers to ways of formatting electronic and print documents in ways that maximize the reader's ability to understand and use the information. The reference in the RFA to Ginny Redish's is a good source of information.
5. Can I submit an application which addresses some, but not all, of the activities listed in the RFA?
As it states in the RFA, applications that do not describe the applicant's ability to respond to all elements of the RFA will be returned to the authors as nonresponsive. We attribute much of the success that CAHPS® has experienced to our great network of grantees, their subcontractors, the SUN (Survey User's Network) contractor, Federal partners like CMS (the Centers for Medicare & Medicaid Services) and OPM (the Office of Personnel Management), accreditation organizations like NCQA (the National Committee for Quality Assurance), sponsors who have agreed to participate in field testing and evaluating CAHPS® products, and many others. But managing a network of this size is often challenging and always time-consuming. Our experience in CAHPS® I convinced us that the most efficient and productive way to organize and monitor the work was to consolidate those who receive CAHPS® funds into "grantee consortia." This limits the "official" points of contact for the Project Officer to each grantee's Principal Investigator and alternate while ensuring that, across all grantees, we have all the necessary expertise to perform the work. One consequence for applicants is that Principal Investigators be excellent managers as well as content experts.
6. One of the report templates listed on Page 7 of the RFA under Report Development and Testing is "CAHPS® for children with special health care needs." The instrument was developed with leadership from the Foundation for Accountability (FACCT). Is it the intent of the RFA that applicants work with FACCT framework or is the template to be largely based on the CAHPS® framework?
The CAHPS® instrument for children with special health care needs (CSN) was developed in collaboration with the CAHMI (Child and Adolescent Health Measurement Initiative). FACCT played a major role in the CAHMI and was critical to the successful production of the instrument. Before we can say whether our collaboration with FACCT will continue, we need to know who our grantees are, so we'll address that question as a team after the awards are made.
7. We're wanting to use CAHPS® statewide in public health for quality improvement purposes. Would it be best for us to partner with a CAHPS® vendor or with a university if we applied for the grant?
You need to review the requirements as specified in the RFA and choose the organization which has the best mix of skills to assist you in addressing this part of the project.
8. Do the current CAHPS® grantees have an advantage over new applicants for an award?
No. CAHPS® products have been available for more than five years and many organizations apart from the CAHPS® I grantees have had the opportunity to use them and learn their strengths and weaknesses. We expect that some of these organizations will consider applying for CAHPS® II funds. But as the RFA states, neither experience with CAHPS® products nor status as a CAHPS® I grantee is a prerequisite for application, though applicants must have experience developing and testing consumer instruments and reports. In order to level the playing field as much as possible, we have made available through the RFA a variety of documents related to CAHPS®. Many of these documents have already been published or have been made available to the public in other ways. Some of them are "working notes" to which only the CAHPS® I team has had access in the past. Our intent is to inform potential applicants as thoroughly as possible about each ongoing or completed CAHPS® I project. Also, the CAHPS® II RFA includes two new areas, quality improvement studies and a greatly expanded reporting task. Since these activities were not included in CAHPS® I, CAHPS® I grantees have no advantage here.
9. Do you foresee that the G-CAHPS® will replace the standard plan-level CAHPS® in meeting health plan accreditation requirements any time soon?
The use of G-CAHPS® for accreditation purposes and other uses will be advised by the committee of users that will assist the CAHPS® developers in deciding the application of the CAHPS® instruments. The health plan level CAHPS® adoption for use as an accrediting instrument came after considerable consultation with CAHPS® users. Adoption of G-CAHPS for accreditation purposes is also dependent on the accrediting organizations such as NCQA and JCAHO (The Joint Commission on Accreditation of Healthcare Organizations).
10. When do you anticipate that there will be enough use made of the G-CAHPS® to have a useful amount of data to employ for benchmarking?
The determination of sufficient data in the National CAHPS® Benchmarking Database of G-CAHPS® data will be based on the availability of data in different sectors, including the privately insured and publicly insured. This decision will be made in consultation with the CAHPS® grantees and SUN contractor.
11. On page 6 of the RFA, one area listed under questionnaire development and testing is "health plans, services and providers." How are you distinguishing providers from services? Also, how broadly should we construe the term "services?"
The term "services" refers to things that happen in interaction with providers and things that happen in interactions with plans.
12. The RFA places the QI (Quality Indicators) work in Phase II. Is it your intention that the proposed QI activities would not begin until year 3 of the project?
You've hit on an area (timing of work) that was tough for us as we wrote the RFA! We think that we'll need to review the timing of lots of different project activities when we meet with the grantees at the beginning of the project. We're also aware that we haven't asked people to "sign up" sites at which these QI projects would be done. So, we're assuming that we'll need a little time up front so that the grantees can do that.
13. Is it important for grantees to obtain letters from NRHCHDR (the National Rehabilitation Hospital Center for Health and Disability Research) and NIDRR (the National Institute on Disability and Rehabilitation Research) indicating their willingness to work with them or is this assumed regardless of the grantee?
Assume that NRHCHDR and NIDRR will work with us on development of products for people with mobility impairments (PWMI).
14. Must applicants have recruited an organization who will work with them as test sites and obtain confirmation letters from them as part of the application?
No, you don't have to sign them up in advance. We do want to know that you're able to recruit sites that you collaborate with and have done so in the past.
15. What are the most frequently requested languages for the CAHPS® instruments, other than Spanish?
It's really varied. In California, they need translations of instruments in different Asian languages. People have also asked for translations in Russian and other dialects of Spanish. We hate to sound like a broken record, but we'll need to consult as a team at the beginning of the project to get some consensus on which languages we'll target. Also, this is a perfect example of where users give us excellent input on their needs.
16. Unlike nursing homes, I didn't see any specific mention of hospitals in the questionnaire development section; are they included under the term "providers?"
When we developed the RFA, we did not specifically target the development of questionnaires of hospitals, but we continue to get requests for hospital survey instruments. We can't say this is a dead issue—issues tend to recycle in this project, but we're not asking for applicants to describe their ability to do hospital surveys as part of their response.
17. What is a good working assumption for the number of members of the advisory committee and the expert panel?
Assume 12 for the advisory committee and 12 for the expert panel. But, as you've heard us say before, these are issues we'll talk about at the beginning of the project.
18. What is the Grantee Advisory Committee?
The Grantee Advisory Committee will be composed of representatives of the key stakeholders affected by the implementation of CAHPS® products. Since this group's role is to advise the grantees (rather than AHRQ), the grantees are responsible for selecting members of this group, setting meeting agendas, making all meeting arrangements and convening the group as often as necessary.
19. In section 4 under budget and related items, it states that the applicant should devote 1/3 effort to each of three areas. I am not clear on what the areas are.
This budget guidance refers to the amount of money you would spend on further development of instruments currently being developed. It means that approximately 1/3 of that budget would be spent on G-CAHPS and Nursing Home CAHPS®, 1/3 on PWMI CAHPS®, and 1/3 on individual provider level CAHPS®. Each of these extends over the first two years.
20. In the RFA you use the term "cultural comparability"; how are you distinguishing it from "cultural competence?"
In our terminology, cultural comparability refers to the development of survey instruments which are neutral to response tendencies of different cultural groups. In contrast, cultural competence refers to the capability of a health service facility to address cultural differences in their clients.
21. The RFA notes testing among those with poor cognitive skills. Is it up to the grantees to propose a method of assessing cognitive status?
Let's distinguish between assessment of cognitive skills to see if people are able to fill out a questionnaire, versus testing of reports and other materials to determine comprehension and ease of use. We were really thinking about the latter throughout most of the RFA. We started CAHPS® I by thinking that a somewhat small segment of the population would require materials for people with poor cognitive skills. When we saw the results of the National Adult Literacy Survey that showed that only eighteen percent of adult Americans can correctly use a bus schedule, we began to realize that there are a lot more people out there with poor cognitive skills than we thought. So, we need to keep this fact in mind as we prepare reports and other materials.
22. The CAHPS® Steering Committee includes PIs and "their alternates." Are the Co-PIs the likely alternates or is this open to other possibilities?
In grants terminology, there really is no such thing as a Co-PI, but we need an alternate for each PI with similar skills. So, it's the PI and alternate who will serve on the steering committee.
23. In light of recent events, is it possible to consider some video-based conferencing as a substitute for some of the cross-grantee meetings?
We would love it! The issue is, not everyone has access to the videoconferencing equipment. So, once again, we'll talk about this after the awards are made.
24. The format and range of the RFA does not fit the format of the typical PHS 398 grant. Has any thought been given to allowing another response format?
At this time, we have not given any additional thought to allowing another response format. However, format and range questions should be directed to our Review Office, specifically Jerry Calderone at JCaldero@ahrq.gov.
25. The RFA states that one goal is to avoid having senior level staff perform tasks that a person with less experience could perform. On the flip side, what is the minimum level of commitment (% time) to CAHPS® II expected from PI and co-PIs?
This is a judgment call. The level of effort to be committed for PIs and Co-PIs should be determined by the grantee's assessment of the project requirements.
26. Re: your comment on the 1/3 budget distribution: are your providing any guidance about what portion of the total budget should go to the development of those 4 instruments?
You should review the requirements and make your best judgment as to the resources needed for developing these instruments.
27. Two free registrations for the CAHPS® User Meeting per grantee are awarded. How much is the registration fee expected to be for other grantee reps?
The final determination has not been made regarding the registration fee. For purposes of budgeting, assume $300.
28. Is "prospective questionnaires" equivalent to longitudinal panel surveys? The RFA refers to "feasibility studies to determine whether prospective questionnaires are desired by consumers."
In the RFA, prospective questionnaires is used to describe CAHPS questionnaires that may be developed in the future.
29. Does the scope of the work involve putting all three CAHPS® products into one RFA? That is, is it the responsibility of one PI to get teams together to address the work involved in all three CAHPS® products (NH, group practices and Persons with Mobility) into one grant proposal? Or, do you mean that a complete competitive proposal is one that addresses one product (for example NH CAHPS®) and its various components (instrumentation, reports, QI and evaluation studies)?
An application must address all requirements of the RFA and not be confined to just one area such as nursing homes.
30. What platforms (PC, Mac, or both) will be supported for the 5 online workshops each year?
The online workshops will be Internet-based, and will support both PC and MAC platforms.
31. What is your anticipated award date?
32. What is the purpose of the steering committee, and what prompted you to include it in CAHPS® II?
The Steering Committee (composed of the AHRQ Project Officers, at least one representative from each of the organizations who join us as funding partners, each Principal Investigator and their alternate) will meet at least annually to consider work for the upcoming year and to decide which products will be developed. During CAHPS® I, funding and/or research partners like CMS and others approached us during different points in our work to discuss collaborations. These partnerships were extremely productive and ultimately enriched CAHPS® products and extended the project's "reach." But we felt that the work could be managed more effectively and dollars spent more efficiently if we considered the universe of potential products and tasks in a more organized fashion and at the beginning of each project year. We hope that the Steering Committee allows us to do that.
33. What start date should we use for budgeting purposes?
Assume May 1, 2002.
34. It wasn't clear whether work on the reporting and QI activities could begin in Phase I or whether you intended those to wait until Phase II. Please clarify.
The reporting activities will begin with a research plan that will span all 5 years of the project. So we need to start talking about that at the beginning of the project. I think we covered your question about timing of the QI projects earlier. So, please check the transcript and let us know if you need further clarification. Also, take a look at section 5 timetable in the RFA.
35. Please clarify how we can get additional info.
The AHRQ Web site contains extensive background information about CAHPS® and supplementary information that may be useful in preparing a response to the request for proposals. It is not mandatory to review this information. Information about CAHPS® I as related to the RFA can be found at http://www.ahrq.gov/fund/grantix.htm and is entitled "Supplemental Information."
36. Are applications feasible that address subgroups of the topics and tasks? For example, CAHPS® MI more than other components.
If your main interest is in addressing a subgroup or only one of the topics, your best strategy is to partner with a group that can address the broad range of the work.
37. Can AHRQ give us additional information on the level of effort for each of the five objectives listed on page 1 of the RFA? Can AHRQ provide guidance regarding the relative level of effort expected on the "research" vs. the "template preparation" components of the reporting task?
While it is difficult to be specific about the balance of research and development activities within a specific contract objective, we can provide estimates of the level of effort expected for the five major contract components:
Moderator: Thank you for participating in today's chat. Since we could not answer all your questions, Chris and Chuck will take questions via E-mail. Please submit them to CDarby@ahrq.gov and CCrofton@ahrq.gov. We will posting any additional Q's and As to the Web site.
Chris, do you have any closing remarks?
CAHPS: Thanks a lot! This was really interesting for us and you asked some good questions. We're interested in knowing how this Web chat format worked for you. Send us an E-mail and let us know: CCrofton@ahrq.gov or CDarby@ahrq.gov.
Looking forward to your applications by November 14, 2001.
Current as of October 2001
The information on this page is archived and provided for reference purposes only.