Skip Navigation Archive: U.S. Department of Health and Human Services U.S. Department of Health and Human Services
Archive: Agency for Healthcare Research Quality
Archival print banner

This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: Let us know the nature of the problem, the Web address of what you want, and your contact information.

Please go to for current information.

2012 Meeting of the Subcommittee on Quality Measures for Children's Healthcare

Transcript (continued)

Transcript of a meeting of the SNAC held September 12, 2012, in Bethesda, MD.

Afternoon Session

Charles Irwin: We are going to return to some voting, but we felt it would be important to make certain here and concur with how we are going to proceed forward. We have decided that we should at least have a discussion or a very brief discussion hopefully, on if we want to use these four categories of assigning measures that Charles developed. I think they are about to come up. We will wait for those to come up.

I think that what we were thinking about is that there would be four categories for voting on measures that we would—the first category would be, we would vote whether our measure should be included in the improving core set or only that the measure would be a good measure for other public and private programs, but not the improved core set.

The third category would be that they would be for both the improved core set and other, and then four would be none of the above. I do not know if this is the right way for us to think about these. We have to come up with the improved core set. We have to come up with are there other measures we want to push forward. It seems to me there might be some that really belong in both categories, and maybe there are some that we do not want to advance. Any thoughts about this, whether this is an acceptable structure for us to view each measure?

Stephen Downs: Wouldn't it be simpler to just vote yes or no on ICS [improved core set] and then on other and then infer the combination if the vote is in favor of both? It becomes two questions instead of four.

Charles Irwin: If it is not an ICS or other, it belongs in none. If it is an ICS and other then it belongs in both. If you just score ICS, and you score other, and you pick a cutoff for each, then you can infer which of those four categories each one belongs to.

Stephen Downs: Maybe I am misunderstanding what exactly you are trying to do.

Charles Irwin: We have an obligation to push forward with recommending a group for the improved core set or not. Is the "not" category—are you implying that if you vote for the improved core set, that those become automatically part of the other category too?

Stephen Downs: It may be my misunderstanding. You are talking about pushing forward the entire set of measures as opposed to considering each measure individually.

Charles Irwin: This is how we would assess each measure.

Stephen Downs: Each measure can fall into each of those categories, but you can determine which categories by deciding individually does it belong in the ICS group or does it belong in the other group. If the answer is yes to both of those, then it belongs to both. If the answer is no to both of them then the answer is no. It is a two-by-two Punnett square. I am just trying to reduce your four categories to two questions instead of four questions.

Denise Dougherty: We have 13 measures to vote on, unless somebody wants to reconsider one of those 13 measures and say no, even though it was voted 7 and above. There is the "none" category. But it seems like to me like we already did the none category, and that everything we voted on so far to get to this leaves us with 13 measures that should go somewhere.

Charles Gallia: And the reason I was suggesting this is that this is affirming it. This is like sorting. There may not be "none." When I construct survey questions, if I do not have that on there, I really irritate people because they want to say that choice was not given me.

Gerard Carrino: Do we have criteria for other? We have some pretty good criteria for pushing it forward for ICS. What are the criteria for pushing it forward to other types of measures?

Denise Dougherty: That was basically the discussion that was question number 10 on your scoring. We tried to have that in the last round of Delphi III where we tried to encourage you to vote on whether it is a good measure in general, irrespective of whether it goes in the improved core set or to another program.

Gerard Carrino: Other then becomes a default if it is a good measure, but we do not want it for ICS?

Denise Dougherty: Yes.

Gerard Carrino: Is there any measure here that would not go to other?

Denise Dougherty: That is the question on whether we have to vote on that explicitly, that yes, they were all highly scored to this point, all 13, but no they should not be used in the future.

Francis Chesley: I would just add to Denise's comment. Coming from my perspective having an affirmative recommendation from this group on a group of measures that may not be ready for improving the core set, but that are rigorous measures and useful for other programs, would be very helpful. One could say by default that is the case, but having an affirmative vote from this group would be very helpful.

Gerard Carrino: But we do not have a list of other programs that it might go to, just other in general.

Francis Chesley: No. And I think that is even beyond our order today to define what other is. Obviously, we have seven Centers of Excellence that are developing measures going forward, and we have found those measures to be both rigorous and useful in a number of ways. One way would be obviously the improved core set. There is a variety of other places where quality measures for children could be useful.

Naihua Duan: In practical terms, would other mean like plans, provider networks? I think we need a little more.

Denise Dougherty: The legislation said other public and private programs, plans, providers, and consumers. That is it.

Charles Gallia: Some ideas that come to mind would be that just as much as I am recognizing the whole world is not Oregon, it is not all Medicaid. But there is HRSA and SAMHSA [Substance Abuse and Mental Health Services Administration], and CDC—I was listing off acronyms. I am just going through the alphabet. CDC, SAHMSA, HRSA, FQHC [Federally Qualified Health Center]—I can see a lot of other applications that are potential and some specialty societies for that matter. Who knows? Education in some components is part of the developmental screen. I can see their applicability to sort of the cross cutting. It does not necessarily mean that the CMS and Medicaid … 

Karen Llanos: I just want to clarify that it is outside of Federal as well. There are programs, private, public health insurers, and private health plans that might also benefit from knowing what an expert panel would think would be good measures. I would think of it that way.

Andrea Benin: When I hear this and when I hear the verbiage that was used as a provider, what I am hearing that other category means is for other payer types of programs to use, not like what do I think would be great to implement in my clinic because this is an important metric that I think my clinic needs, and I kind of want to improve things. What you are saying is, is this a metric that we want another payer to take up?

Karen Llanos: There is actually no leverage that we have. It is more up for informational purposes. And I think it is sometimes helpful to know in a world where there are lots of measures, what are some that have been through a multi-stakeholder process. In consideration of the improved core set, these also came to the top.

Denise Dougherty: And it is other payers, other programs, mental health programs, other plans, other providers, and other consumers.

Karen Llanos: But sort of in the publicly reported type of …

Participant: This group thinks, for example, that if you are a clinic that is treating kids with autism, we do not have a measure for that, this would be a good quality measure.

Francis Chesley: I guess the three of us are trying to reinforce, we do not want to put too fine a point, to be too specific about other uses of rigorously developed measures deemed by this group. That is not even for us to come up with that laundry list, so much as to say that in the world of quality measurement and improvement, there are many levels in which quality measures may be useful. And having some sense, as one who wants to use a quality measure that it has gone through some kind of a multi-stakeholder review such as this, would be helpful. As I am looking at the grocery list of all the available measures that I could use, these will be useful for my purposes.

Andrea Benin: But then I would, I have two paired questions that I have in my mind as you are saying this. One thing I would say is, isn't that what the NQF [National Quality Forum] process should be that that is where that, if I were to look for a measure actually that it went through some kind of a process like a peer review, is this intended to take the place of that?

Denise Dougherty: The interesting thing about what this group has done is that the desirable measure attributes are more specifically focused on children's issues than the NQF criteria would be, for example. The different thing about this group is that it is a legislatively endorsed process for children's health care quality measures specifically. That does not mean it substitutes for NQF. Certainly, any measure that this group recommends or does not recommend can go to NQF separately. The legislation did not call for any going to NQF. But it did call for us to develop measures that could be used by others.

Andrea Benin: Then on that timeline when there is a list of now "other" measures, where do those go? I do not remember that part of it. Do they go into some report or something?

Denise Dougherty: That was a question I asked in the beginning. Maybe if we really feel strongly, if you all really feel strongly about that, you can help us devise some, think about some strategies, resources permitted that would allow—all the legislation says is, disseminate those measures. Dissemination can be really active, or no resources were provided for that, really passive. We can just put them on the Web site.

Charles Gallia: One suggestion was that we create T-shirts that I am "other," only an orphan for each one of the measures. It would not really have homes or ownership or support other than by the submitter or the proposer. They are the ones that would be responsible.

Denise Dougherty: But I could see, for example, Betsy saying taking a measure to United Healthcare or the Blues in Massachusetts, or whatever and saying, this is a child health-specific measure that was recommended by this group of child health experts. What they do with it, the Federal Government has no control over, but that could happen.

Charles Gallia: There was an example of one that we were reviewing, and part of our metrics for the coordinated care organizations in Oregon is a kindergarten readiness assessment. That is not one that we proposed as being part of our measures that were reported through the health plans and health systems. But it is one that is actually administered in the educational setting. It has importance and relevance, and we wanted to make sure that somebody was paying attention to it. And it was sufficiently reviewed for its sensitivity and reporting levels that we thought it was good enough to go forward. That is how we ended up with it as another kind of group.

Participant: I think I have enough information to make an internal decision about how I might think about these things. Is that helpful?

Andrea Benin: I think the thing that is a little bit unclear is that we are voting on something like an unknown purpose. It does not feel exactly right to me, which means I will be conservative in my point.

Charles Irwin: I guess the other option would be for us to not have this classical. Maybe we are making it too complicated. Maybe we should only be choosing what is in the improved core set and then just by default, all the other measures that muster sufficient vetting and discussion that they met criteria, and we would include them. We have 13 measures.

I think the biggest question right now for us is how many measures should we be moving forward for consideration for the improved core set. I know that our colleagues in CMS would like us to be somewhat judicious and thoughtful in adding 13 more to the 24 already in existence. I guess one option would be, do we establish that we will only choose five. I do not know. Or do we just look at them all and vote one by one and say, oh my gosh. Now we have 12 because only one we felt should not go forward for the improved core set. Then we go back. We say now we have 12. Let's vote again. And we will get rid of six this time. We sit here as the co-chairs, but all of you are a part of this committee, and we would welcome your discussion and debate on how we do this.

I think this is pretty important right now. It is very important on what we choose out of these 13.

Stephen Saunders: I would suggest that we start and see how it goes, and we just vote at this time. ICS, yes or no. If it comes out 12, then we will have to think about what we are going to do about it. If it comes out 3, well maybe that is fine. We should just start and see how it goes. The whole list, the whole 13 could go to other.

Charles Gallia: That is one of the decision points that we should take a motion on and do that, and then we can dispense with the other one. What I was just mumbling was what we should do then is entertain a motion that would say that the entire 13 that made it through these processes we would consider recommending by default for other, and then having a second, and then formalizing that, and then we are done with it. We can eliminate it from the decisionmaking moving forward and then just do as you suggested.

Stephen Saunders: If you want, I will make that motion that we should consider all of the 13 for the other.

Charles Irwin: Do people need to vote anonymously? We should. We have to have a 1-2 vote.

Denise Dougherty: While we have a break, just a couple of logistical things that if you are not registered at the hotel and have not had a dinner here, you have to pay for your own parking.

The hotel is unable to reimburse parking.

The other thing is that there is a dinner tonight for the SNAC members and the COE representatives who are here in the hotel private dining room if you would like to come to that.

Thank you.

Charles Irwin: Before we vote on the individual measures, are we going to see the existing core set, or at least make certain that people are aware of what is in the existing core set? If you are not, it is in section 5.1. Can you tell us how these are grouped for us?

Participant: If you see an X in a column that says initial set, it is in the initial core set. If you see a measure number and an X in the most far right column, it is up for discussion. It is 1 of the 13 measures that have passed by 1, 2, or 3. It starts with prevention, health promotion measures in different categories, and then it goes on to the acute measures. I did not break these out into some of the subcategories you have seen as upper respiratory infections, but you can see which are acute, which are inpatient, which are patient safety, imaging, asthma, ADHD [attention deficit/hyperactivity disorder]. And then the chronic measures come up: mental health, diabetes, and substance use and abuse. And then finally there are the cross-cutting measures. The two that pass through Delphi round III this morning and then the availability of services measure and family experiences of care measure that are in the initial core set. Do you want to go through this again slowly? You drive the bus.

Denise Dougherty: I am curious why there is not a C-section measure listed under the initial set. It is not paired with the one that is in the past. It is a little hard to tell then that we have two measures on the exact same topic. You have to read down. Is it possible to move the row? Just put an X for number 056 next to the initial set C-section measure so that we know that there are two of them. Are there any others like that?

Clint Koenig: Yes, Denise, there should be 55, which is elective delivery as well as cesarean section. They are thematically grouped.

Denise Dougherty: Put the X there and then just put number 056 so we know which number it is. There are some in the adult core set that we did not get. Sorry. We did not give you a complete guidance. Well, maybe we can just put in X. When it says initial set, we can put two X's. One is in the CHIPRA set and one is in the adult set. That color-coded table? Where is that? It is just PCL1 that is missing. PCL1 is C-section, elective delivery. But it is not in the CHIPRA? It is not in the existing.

Everybody has that in their binder in tab 4.2. That provides some more details. But this was supposed to be a summary of that because that one is a little confusing. 5.2. Sorry. We are set. The next thing is, we are going to vote one by one on whether they should go into the improved core set or not. Yes or no.

Charles Irwin: I think we have to come back to these measures, the ones that are duplicative, and really speak to how they are duplicative. Karen, could you possibly speak to the ones that are duplicative.

Karen Llanos: I have text specs for the initial core set. For the C-section one that we currently have, it is the percentage of women who had a C-section among women with first live singleton births at 37 weeks of gestation or later. I do not have the text spec for the one that we just voted on, but I remember the gestation period I think was the same, 37 weeks or later. This is drawn from State vital records alone or merged with discharge diagnosis data.

Denise Dougherty: Are we sure that 056 did pass Delphi I, II, or III? Yes, I guess we are. That was the Childbirth Connection measure. Thanks.

Charles Irwin: Duplicates. Do we want to vote on this one now, or do we want to hold until we go through them all? It is up to the group.

Karen Llanos: Denise, I do not know if you had the description or someone at RTI for the new one, but it might be helpful to hear what the description is for that one to see if there are any big differences.

Denise Dougherty: Okay. That is a good idea. I have something here, the basic description. This measure assesses the number of nulliparous women with a term singleton baby in a vertex position delivered by C-section. This measure is part of a set of five nationally implemented measures that address perinatal care. I have to open up the summary. Denominator statement is patients delivered over live term singleton newborn in vertex presentation. Ten data elements are used to calculate the denominator. Admission date, birth date, clinical trial, discharge data, gestational age, ICD-9, other diagnosis, other procedure, principal diagnosis, principal procedure, and parity. Numerator statement is patients with C-sections with ICD-9 CM principal procedure code or ICD-9 CM other procedure codes for C-section as defined in Appendix A Table 11.06, available at the Joint Commission manual. This is the Joint Commission measure.

Karen Llanos: The current one is maternity quality care collaborative measure. The eligible population Medicaid, data sources I said as vital records, discharge, diagnoses data. The live births add or beyond 37 weeks to women having their first delivery and are singleton and are vertex presentation parameters are described in administrative data sets. The numerator is the proportion of denominator that had cesarean births.

Denise Dougherty: The California measure is different from the Joint Commission measure. The question is which measure is better. Should the Joint Commission measure replace the California measure that is already in the initial core set?

Clint Koenig: Could I ask for some clarification. Is this 055 or 056 we are discussing?

Denise Dougherty: It is 056.

Stephen Saunders: I guess it would seem to me that if they are substantially the same, and I am hearing that they are, I do not know that, but it sounds like they are, that we ought to go with the existing one because States have already invested the effort to program that measure. And the point here is to monitor C-section rate, and obviously we want to reduce it when it is appropriate and so on. I would suggest that we eliminate, vote against this one, adding the new one, and leave the old one unless someone can demonstrate that it is significantly more valuable. I doubt that it is.

Gerard Carrino: I would further recommend actually to go back on something that I was thinking just a few minutes ago, and that is also not putting in the new one if we deem it to be substantially the same as the old one, and if the old one is superior to the new one, not putting it into the other category because it would just be confusing. This would be the only exception to that rule as far as I was concerned.

Charles Irwin: We found a reason to have a "none" category.

Charles Gallia: Just a point of clarification. To suggest that this measure would replace something that is in the adult core set is really beyond what I think the charge of this organization is. Does that make sense?

Participant: But that measure is in the children's core set.

Participant: What I heard was a discussion about if there is a duplicate or if it is in another set and this is a superior measure, that we have boundaries, but we can recommend.

Charles Irwin: Should we vote? What we are going to be voting on is the measure that was brought forward through the Delphi process that is either going to be included or not included in the core set. That is the vote first.

Participant: But then we are going to vote on whether it goes to other?

Charles Irwin: I think we have to do it separately. Two votes. We will pull the others at the very end; 1 down, 12 to go. We are going to actually wait until the end, and we will do them all at the end. That is what a recommendation is here.

Let's go back to that. Just see if we have any more duplicates.

Karen Llanos: I think it is PCO1. That is the other duplication—PCO1, elective delivery. It is the Drake commission measure. It is part of the initial core set of adult measures that was just released in January this year. States will start reporting in 2014. Other States can chime in. But I am not sure the differentiation matters that much. States pull from both core sets for their Medicaid and CHIP population as needed.

Denise Dougherty: I guess the question for this one is whether—I think we discussed this on some phone call or other. Just because it is in the adult core set does not mean it only gets measured among adults. If there are adolescents who give birth, they are included.

Karen Llanos: That is what the eligible population is.

Denise Dougherty: The eligible population is—I think it starts at age 8, I think you told me.

Karen Llanos: It is a pretty wide age range.

Charles Irwin: It is 8 to 61 or something. I am serious. We can assume that this measure is being adequately covered in the adult population in the adult measures.

Karen Llanos: It is who reports into CMS, and it is the Medicaid program. We do not differentiate in the Medicaid program as to whether it serves children or adults. And sometimes, it is the same data people who are plugging in. I do not think it differentiates. That is my opinion because it is collected by the Medicaid program, not particular people who are focused on children or adults. It is used for the ultimate purpose of understanding the quality of care provided for pregnant women.

Denise Dougherty: But what about the CHIP program?

Karen Llanos: The CHIP program could—that would be the one piece where it would. As of now, they do use two different reporting templates. In the future, we would just use one reporting template.

Denise Dougherty: That might be a reason to add it.

Karen Llanos:—CHIP programs. I am looking at Marsha. There is only a handful of them that would have to collect data for that particular population. Everyone else would be blended into the Medicaid program data.

Charles Irwin: Karen, can you clarify for me? When a girl becomes pregnant—of course, in our State she automatically goes on Medicaid, but evidently it is not everywhere.

Karen Llanos: I think pregnant women can be enrolled in CHIP programs too.

Charles Irwin: We would miss those girls that were in the CHIP program with the adult measure.

Karen Llanos: We are actually building our adult reporting template right now. One of the questions we ask is does this population represent Medicaid and CHIP, and in parentheses, we write pregnant women enrolled in CHIP. It would actually be the same question. We are in the process of building the reporting template that would actually help a lot of these questions. I just realize it is one of the things that we do call out, pregnant women in particular, who would be enrolled in CHIP.

Charles Gallia: We have a prenatal care program that is funded through CHIP, and  I would imagine we report the measure, that we would just use the age specifications, not necessarily the insurance.

Karen Llanos: Are you talking about the CHIP annual reporting pieces?

Charles Gallia: Yes.

Karen Llanos: That would be the difference that I see.

Marsha Lillie-Blanton: The core set of measures whether for children or adults are only collected on Medicaid and—I am thinking through this right now. It has to be a CHIP expansion program, or CHIP Medicaid expansion program, or a combination program. If it is a standalone program, we do not have a systematic way of collecting those data. We certainly encourage the data to be collected, but we do not have a systematic way of collecting the data if it is a standalone program. We encourage it, but we do not have the same leverage because the core set of measures at least for children—I do not think it is a big issue because—you are raising an issue I have not thought a lot about, but we do not do as much outreach to CHIP standalone programs, but certainly they are part of our network that we reach and communicate with. It really varies.

But I can tell you just in terms of reporting, we have more of a history of reporting from CHIP than we do for Medicaid because there were four core measures collected. They were not called core measures, but four of the measures collected by CHIP became a part of the core measures collected by Medicaid. I am not sure I am helping much.

My bottom line is going forward we are trying to make fewer and fewer distinctions between Medicaid and CHIP. We are trying to say there is public coverage, and there is private coverage. There is coverage by the exchanges. Medicaid and CHIP, while they are separate now, by the time we move to 2014 and 2015, our goal is to create within the State systems coverage. I think the question you are asking is do we need to have an elected delivery in the children's core set because we want to make sure that CHIP is covered, and I just do not think it is a relevant or key issue.

Charles Gallia: The resulting data would allow those CHIP standalone programs to make a comparison between other CHIP standalone programs for the CHIP-eligible population that might meet the elective delivery criteria. If you are going to try to do something with, it could be a little bit challenging because it is a pretty narrow segment as part of being a core measure. It is going to be fairly narrow. Or you are going to have to produce two measures.

Marsha Lillie-Blanton: Let me just say, I feel like you all are raising issues that we are trying to resolve now. I do not think having the same measure in two sets would be helpful. I think it would be confusing if you had the same measure in the child core set as you had in the adult core set. It is both the Medicaid program and the CHIP program that are collecting these measures. And it is on the population that is enrolled in one program or the other. In CHIP, there is a very small group of women. I know this only because I just looked at the number, but in the last 5 years it is maybe 400,000 births occurred in CHIP in the last 5 years. It is very small. That is an artifact of history because States were allowed to enroll women in CHIP, but Congress stopped States from enrolling adults in CHIP. Going forward, you will not have many women, but you will have adolescents. But you will not have adults. This is a decision you all can make.

Charles Irwin: Any more discussion about this measure? Then I think we should entertain nominations of whether we should consider this as pushing this forward or not pushing it forward.

Stephen Downs: I move for the vote.

Elizabeth Anderson: Second.

(Vote taken)

Charles Irwin: Okay. We have 11 to go. Any other duplicative measures? I forget. Is there one other in the—

Denise Dougherty: Can you make that larger?

Participant: Can we know what the initial core set question is?

Denise Dougherty: Do you want to explain the latest version of the CLABSI [central line-associated blood stream infection]? It is going to be the CDC definition, CDC NHSN [National Healthcare Safety Network].

Karen Llanos: It is currently a CDC measure. It is the pediatric central line-associated bloodstream infections. The rate of central line-associated bloodstream infections in pediatric, in neonatal intensive care units, during the period for selected surveillance. The central line—the CLABSI infection is an infection –you all know this - in a patient that had a central line inserted within the 48-hour period before onset of infection.

I will say this has been one of the harder measures for States to collect, mostly because it is drawn from hospital data that normally are not broken down by payer. We had a work group where Denise and Charles participated in CDC. We also had lots of experts in the field. We will be trying to get these data through CDC, and it will potentially not be by payer. It is a very difficult measure for States to take Medicaid-specific data on.

Denise Dougherty: The one that is proposed is an AHRQ pediatric or neonatal quality indicator, which is based on hospital discharge data. I could through all the denominator and numerator details. But the reason this expert group we had did not recommend using this measure instead of the CDC measure was because of the trouble and unreliability of coding in the hospital setting. But I can redo all the details of this one if you would like. Raj, I do not know if you have done any work on this particular patient safety—

Karen Llanos: Charles is the one State that reported—on CLABSI. I do not know, Charles, if you want to add anything in terms of Oregon's experience.

Charles Gallia: It was enormously complicated. It is an important subject matter area. The measure itself—the only reason that we have the potential for identifying payer is because there is a separate supplemental CDC grant that is allowing us to do a little more drill down. But other States are not going to have that opportunity. And then in the conversations of the work group, it was clear that Dr. Mangione was—that this was a focal area that is more broadly centered on the hospitals, and that the real action-oriented accountability is appropriately there as opposed to the payer type recognizing that if—with the weaknesses of the existing measure and as a surrogate that one that is being proposed could be something the States could use in the meantime. But I would not suggest that it be a core set measure. This can't be produced by the States on an ongoing basis until there is more clarity on measurement specifications.

Glenn Flores: If I recall this measure correctly, I had a fundamental issue with it because I believe it includes all bloodstream infections, including sepsis—regardless of the cause. It does not separate out nosocomial. I do not think that is a good way to measure quality for this indicator.

Denise Dougherty: It includes Staphylococcus, MetSOs, I do not know what these things stand for, MRSA [methicillin-resistant Staphylococcus aureus], septicemia, gram-negative septicemia, E. coli [Escherichia coli] septicemia, Pseudomonas, serratia, gram-negative septicemia, and disseminated candidiasis, or any secondary diagnosis for NB [newborn] septicemia or bacteremia of newborn.

Glenn Flores: In other words, if you have a population that has a high prevalence of group B strep, which is a lot of the underserved populations, it is going to look like you have a poor quality outcome here, and that is just going to be a red herring. I do not think that is a good indicator.

Charles Gallia: Furthermore, there was an issue about transport and facility base so the initial diagnostic activity—if there is acuity transfer at the hospital level, then it is going to confound some of it.

Rajendu Srivastava: Just a couple of comments about the central bloodstream infection from a hospital perspective. I agree. It is a very rare outcome. It might not be that important at the State level or for a prevalence population. Just remember that even if you are covering a State, there are just a few kids that will get admitted to the hospital, but that is where most of your cost is going and morbidity and mortality. I just want to caution the group that while prevalence measures are critical, the things that affect just a few children are also really important.

Number two, Glenn, getting to your point, there are a number of papers that have used—I did not read the initial core set CLABSI rate. Shockingly, it was not in my packet. I do not know how that was done. I do know other CLABSI rate papers that are published, and they do it actually very nicely looking at groups. Pediatric oncology patients should not get these infections because they will die when they get these infections. It is used in improvement work in that area.

Glenn Flores: Raj, I am less concerned about the CLABSI. I am more concerned about 14, which is this neonatal bloodstream infection. It is just too broad to be able to be used as a quality indicator.

Alan Spitzer: If I could comment on that. I do not think that that is too broad at all. I think this is one of the cardinal efforts now nationwide in neonatal intensive care. There are several I am aware of State collaboratives ongoing now looking specifically at that particular indicator. This has probably been the most important change in newborn intensive care in the last 5 years, as the reduction that has occurred by focusing on that issue as an outcome measure.

Glenn Flores: But as a neonatologist, I would think you would agree. You are going to have a certain background noise of group B strep, E. coli—

Alan Spitzer: No, because those things are early infection for the most part, and this refers to late onset infection. These are usually babies that are in the NICU [neonatal intensive care unit] beyond 7 days of age. There is still a lot of late onset GBS [group B streptococcal septicemia].

Glenn Flores: No. GBS is an early onset infection at the present time. There is very little late onset GBS.

Alan Spitzer: That is why we do—work up in infants is because you still see neonatal bloodstream infections out to 2 months.

Glenn Flores: Of course, you see infections. But we are talking for the most part about low birth weight, hospitalized babies. This is an extraordinarily important measure.

Alan Spitzer: Neonatal bloodstream infections. It is not limited to any one population. It is an overall quality measure. We are actually talking about it—that is why I think it is not a good indicator.

Marsha Lillie-Blanton: I think it would be helpful for us to look at comparing the two because it is unlikely in our core set that we would have both. It would be helpful for me to hear the discussion of is one better than the other. I have my own views, but I would appreciate hearing from this group.

Participant: I was actually on the working group with Marsha and Charles. There have actually been comparison studies that have looked at using ICD-9 administrative-based algorithms like the AHRQ measure and compare them to the CDC methodology, which actually counts line days and uses infection control people in hospitals to actually trigger when an infection gets counted. Surprisingly, using the administrative measure grossly undercuts the numbers of infections. When you look at hospital real infection rates as collected by the CDC methodology, and then you use the administrative methodology in that same data set, you will see pretty severe under counting of hospital-acquired line infections. I have my own opinion about which one is better, but I think you should know that.

Charles Gallia: Following on that, with the administrative data reimbursement policies that will have an impact on—even their appearance in administrative and claims data over time—so the distortion that you might see as a result of the self-initiated claims submission versus the observation and validation steps that—this location of the site. It is very specific. When it is produced, you know for certain that there is no question about the occurrence. It is just the challenge or the weaknesses that it does not reflect Medicaid. But it is still a good sound measure.

Clint Koenig: I do not know if you have seen the methodologies for the proposed in the current core set. Can you maybe comment on which one may be more feasible to implement?

Karen Llanos: I do not have the text specs for the—but I think Rita outlined—of the two measures. One of them draws from—the AHRQ one draws from—which is probably more readily available.

Participant:—said that the AHRQ one, which draws from administrative data might be more feasible, but not as valid, whereas the CDC measure draws from observation which is a little bit harder to do, but still feasible and more accurate.

Andrea Benin: One of the things that bothers me about this discussion is that the States—we report through NHSN the stuff to the States to many of the States. I do not know if it is all of them.

Participant: 25.

Andrea Benin: Half of the States already collect this kind of data through NHSN. It seems to me just—to the proposed and not an additional different measure of inferior quality to put into a mix of what is out there around central line associated bloodstream infections. It seems pretty straightforward that that is not a good alternative.

Denise Dougherty: We had no control over what got submitted. People did vote on this measure as it was presented. That is the process.

Andrea Benin: Then with regard to the high-risk neonates measure, the other measure number 14. Which one are we talking about? Were we talking about the other one?

Charles Irwin: Central line infection.

Andrea Benin: We are only talking about central line.

Charles Irwin: Versus the neonatal measure. That is what we are trying to make a decision. If the neonatal measure should come forward—we would not say it would replace it because that is the decision of the CMS. But the question is do we bring it forward for CMS to consider for a part of the improved—

Naihua Duan: Isn't it possible to consider a tiered approach like in survey research, very often we try to get the complete survey. But for those who did not want to do the complete survey, we might offer them a short version to at least get some basic data. It sounds like with a tradeoff between feasibility and I guess validity. That is a possibility. If you can do the CDC version then great, you should do it. If you really cannot do the CDC version, could we offer the administrative version as a backup? I guess somebody has to do some work to try to calibrate between the two. That way we might have a backup to get some data from those who cannot do the CDC version.

Charles Irwin: I guess what I was hearing from Rita is that they are different measures.

Participant: My concern is that if one organization is using—and the other one is using a more valid measure, that organization using the administrative measure—we really need to think this through—

Diane Rowley: First of all, I need to make sure I am clear. The neonatal infection one would count any bloodstream infection and not the nosocomial infection?

Denise Dougherty: Not all of them, but there is a long list. It is a different—

Participant: All central line-associated infections.

Participant: No. All neonatal—

Alan Spitzer: Could we have those criteria back because I read this originally and maybe it is my error? Not the infections, but the population that was being looked at in neonatal because I thought it was a hospital-based low birth weight population. And it is. But this is the low birth weight in hospital population that is being looked at. That is a very important population. I cannot speak more strongly for adopting this measure because it is so critical now in newborn intensive care. And I think the reason if there is under reporting, it is because people have not been forced in many cases to report this particular measure. It is one of the most important factors that prolong hospitalization in neonatal intensive care units.

Diane Rowley: I hate to get picky, but under the denominator, the last bullet says age and days less than 2 and transferred from another health care facility. If it is less than 2 days, you are not picking up—presumably you are picking it up one time. It would not pick it up in the hospital birth in the hospital—

Alan Spitzer: I think that was the intent there, but I do not know that that is worded correctly. I remember reading that also and that it did not make sense to me.

Diane Rowley: I just have one other thing I wanted to mention. I realize that the CDC measure does not give us information according to payment. But I think that it is better to have a really good measure and worry about the payment. I think that can be collected perhaps from the general hospital record. I would be in favor of choosing the measurement that has been validated and proven to be very useful as opposed to having an under count just because we want to know by pay status.

Glenn Flores: I want to follow up on Alan's comment because I am still confused. If you look at age in days less than 2 and transfer from another health care facility, the birth weight greater than 1,500 grams, it actually is not limited to low birth weight, is it? It looks like it is a broader population. I think that is an important distinction.

Alan Spitzer: I think there are wording problems in this. I think it should be birth weight less than or equal—

Denise Dougherty: Could RTI pull up the actual submission because that has something different. It has the denominator State and all newborns and—with birth weight 500 to 1,499 grams or gestational age between 24 and 36 weeks or birth weight greater than or equal to 1,500 grams and in hospital death or OR [operating room] procedure or mechanical ventilation or age in days less than 2 and transferred from another health care facility. Is that the same?

And then it has denominator exclusions. Exclude with the principal diagnosis of sepsis or secondary diagnosis code present on admission. This does include present on admission, which is something you do not get with the AHRQ quality indicators with birth weight less than 500 grams, with length of stay less than 2 days, with missing data for sex, age, quarter year, or principal diagnosis. That is I think a little clearer than what we saw.

Andrea Benin: Do we know how this metric compares with the Vermont Oxford metrics because this is typically done through a Vermont Oxford registry, and these kinds of things are typically addressed with rigorous data collection and sort of existing registries.

Denise Dougherty: But Vermont Oxford is proprietary. I have never seen data by payer.

Alan Spitzer: I think the intent was probably hospitalized critically ill neonates here and try to include whatever that is. For the most part, it is going to be your low birth weight baby.

Mary Evans: I nominate this for voting.

Charles Irwin: We are going to vote on this. Yes or no?

Participant: Second.

(Vote taken)

Charles Irwin: I believe this ends the three that overlapped. Am I correct? And none of them came forward. Is that correct?

Participant: Correct.

Charles Irwin: For the improved core sets. We have not decided whether they will be in the other, but that we go back now to the 10 measures that there is not overlap or any other competing measure at this point. We are not taking nominations from the floor right now. No confetti, no balloons, and no Bill Clinton. You are all just going to have to endure the pain and suffering of us; 10 to go. Shall we begin this process of the 10? Depending on how individuals feel, we will bring each measure up. The question is do we want to—how much discussion do we want about each measure? I will leave that up to the floor to make that decision. We have 10 left. It is currently 2:40.

Participant: Should we try to finish by 3:30?

Charles Irwin: We will not allow more than 10 minutes discussion for each measure at the most. Preference will be 5 minutes. Maybe we should set the clock. Let's go back to the measures.

Participant: The table so we can see the rest of the measures.

Charles Irwin: We really want to see where the measures, not that they are competing measures, but other measures in that category. Can we get rid of the ones that we have eliminated? The initial core set too?

Participant: No. I think we should leave the ones in the initial core set because it does give a flavor of what is being covered in this category. You will read to us what is in the healthy term newborn.

Denise Dougherty: The brief description is percent of term singleton live births excluding those with diagnoses originating in the fetal period who do not have significant complications during birth or the nursery care. I am sure you want more details than that. The numerator is not well said actually. The absence of conditions or procedures reflecting morbidity that happen during birth and nursery care to an otherwise normal infant. The morbidities may or may not have clearly been the result of medical care. It is the absence of a long list. Birth trauma, an injury, fetus or newborn affected by other complications of labor and delivery, subdural cerebral hemorrhage, subgaleal hemorrhage, clavicle fracture, other skeletal injuries, spinal cord injuries, facial nerve injury, brachial plexus injury, other cranial peripheral nerves, other specified birth trauma, hypoxia, asphyxia, severe birth asphyxia with neurological involvement, mild or moderate birth asphyxia plus or minus neurologic involvement, HIE [hypoxic ischemic encephalopathy] 768.7, unspecified birth asphyxia, congenital or infantile.

Then there is the category shock resuscitation and complications. DIC 776.2, NEC 777.5, shock hypertension, renal failure, then procedures arterial catheterization, umbilical venous catheterization, TPN [total parenteral nutrition], gastrostomy, gavage feeding, cardiopulmonary resuscitation.

Then there is a category of respiratory. Pulmonary hypertension, RDS [respiratory distress syndrome], meconium aspiration with respiratory symptoms, clear AF aspiration with respiratory symptoms, pneumothorax, pulmonary hemorrhage, primary and other atelectasis, TTN [transient tachypnea of the newborn], other respiratory problems after birth, apnea, cyanosis, respiratory arrest or failure, hypoxemia, aspiration of stomach contents. And then under respiratory procedures, noninvasive mechanical ventilation without delivery through endotracheal tube or tracheostomy, bilevel airway pressure, BiPAP [bilevel positive airway pressure], CPAP [continuous positive airway pressure], mechanical ventilation not otherwise specified, noninvasive positive pressure, noninvasive PPV, NPPV [non-invasive positive pressure ventilation], delivery by noninvasive interface, facemask, nasal mask, nasal pillow, oral mouthpiece, oronasal mask, other respiratory therapy. Mechanical ventilation delivered through endotracheal tube or tracheostomy includes a long list. Fourth digit is for duration, inhaled nitric oxygen, chest tube.

Infection category. Congenital pneumonia, septicemia of a newborn, bacteremia of the newborn, severe sepsis. Neurologic complications. Intraventricular hemorrhage, subarachnoid hemorrhage, seizures, other unspecified cerebral irritability, coma and cerebral depression, periventricular leukomalacia, cardiac arrest, encephalopathy, cerebral edema. Neurologic procedures. Computed tomography of the head, other tomography of the head, MI of the brain and brainstem, EEG [electroencephalogram].

Then there is a category called disposition-LOS [length of stay] , neonatal death, neonatal transfer out, LOS [length of stay] greater than 5 days, hemolytic disease due to ABO incompatibility, isoimmunization, phototherapy, housing household and economic circumstances, family disruption due to child and welfare, family disruption due to child and foster care, or in the care of a nonparental family member. And the time window is the initial neonatal birth complication only.

Then there are a number of numerator exclusions and a number of denominator exclusions. I will just read you the categories. Congenital anomalies—you get the sense.

Rajendu Srivastava: The question I have, which I could not arrive at, but maybe somebody on the SNAC who read this came up with this, is the list is great. I went through and there are some things missing on the list as will be the case whenever you create a list. The question I had is I thought it is missing this and this is, has this list been studied in a population and validated? At some level, when you are picking a quality measure, it is nice to know the list has been tested, vetted, tried in a couple of different settings and is capturing the "healthy" population pretty well. It will never be perfect. I could not get that sense by reading what was given to me in the materials for the measure. Does anybody know the answer to that question? Has this been studied, validated, or is this just a list? I am ready to vote.

Denise Dougherty: I am looking to see if there are any citations.

Stephen Downs: While Denise is looking, I am going to comment that it was my observation that a number of these things I thought should not be on the list, just based on face validity, things that really—it says at the beginning it may or may not be directly the result of medical care, but there are many of the things listed there that there is no evidence that they have anything to do with medical care.

Glenn Flores: I have to agree with Steve. This is a list that I have no idea where it came from; whether there is evidence driving most of this is a good question. I do not know how you could say what a quality measure is. For example, housing, household, and economic circumstances, family disruption to child welfare custody, that is population by population. It varies dramatically. This just really does not seem like a good avenue for us to go in terms of this being a quality measure.

Rajendu Srivastava: Or I guess what I would say is—I know Denise is still looking. If there is no such documentation that we can find, what I would say is to the people if they are going to resubmit, this is the kind of information we need to know in order to make a more informed decision.

Denise Dougherty: This being, we need to see some testing experience, validation.

Rajendu Srivastava: Yes. That would be helpful. I was going to say it much more eloquently—

Charles Irwin: My concern also is many of these issues are—I am not certain how we can improve them as part of the health care delivery system. I do not think it fits.

Participant: Can I call the question?

Participant: I second.

(Vote taken)

Charles Irwin: Okay. Next one.

Denise Dougherty: The short version is the percentage of female adolescents 13 years of age who had three doses of the human papillomavirus vaccine by their 13th birthday. The denominator is female adolescents who turned 13 years of age during the measurement year. The numerator is documentation of three doses of the HPV vaccine administered by the 13th birthday. Data sources are administrative claims, paper medical record, and electronic medical record. Numerator exclusions are not reported. Denominator exclusions are adolescents who had a contraindication to the HPV vaccine.

Karen Llanos: Just for context setting, we have as one of the measures adolescent immunization status. It does not include the HPV vaccine.

Carole Stipelman: Why is maleness an exclusion factor?

Denise Dougherty: Males were recently added to the recommendation. It takes a while for quality measurement to catch up.

Charles Irwin: The recommendation is not—I cannot think of the name of it. It is an option. I forget the term. I do not think it would be good to include that as a measure. Are we ready to vote or not? Let's vote.

(Vote taken)

Denise Dougherty: I think it was supposed to be 70 percent yes.

Charles Irwin: We are going to let something fly if it is 50, 49?

Participant: I have faith in democracy.

Charles Irwin: I think it should be a minimum of two-thirds.

Francis Chesley: I hate to bring the process issue up, but we did not actually decide what the cutoff point would be at this point. We did 70 percent to get into this level, but at this tier, I think we needed to actually set the bar. We just did not do that. And it occurred to me that we should do that once I saw the vote.


Charles Irwin: Someone said 50. I think 50 as a cutoff is too low, but 51 or 49 does not make sense to me. I think it should be at least 60 or 65.

Charles Gallia: 51 percent, 67 percent, over 70 or greater.

Charles Irwin: What is this?

Charles Gallia: You cannot do it that way. It has to be greater than 50 and then it has to be 67 percent, two-thirds. They just say it. Not greater than.

Charles Irwin: I do not think you can vote for this way. We will not get an adequate response here.

Charles Gallia: For the decision affirming would be—the majority 51 percent, 67 percent. That is the threshold. And then if you want to say 70 percent, that is fine. And then we are going to say this is the cutoff point.

Participant: Are we going to have a runoff vote?

Charles Irwin:—get one or two. I think we should get rid of three here.

Participant: What percent of this vote—

Charles Irwin: I think we should say the majority rules here. It is either one or two.

Participant: I move that we select one of these.

Charles Irwin: Should we do 66 percent rather than—

Glenn Flores: Can we show the time lapse? I think some people might be missing—the time is cutoff on the screen.

Charles Irwin: I think we should do 66 percent. Can we vote?

(Vote taken)

Charles Irwin: We need to come back and vote on the HPV vaccine one again. This is post hoc. Fine. I do not look forward to voting any more than I have to. Does anyone want to re-vote on the HPV vaccine? We are comfortable with what happened.

Let's move on. I think we are at eight now. Are we at eight? I think we are at eight.

Denise Dougherty: The short description of this one is admission rate for gastroenteritis in children ages 3 months to 17 years per 100,000 population. This is another one of those RQIs with long lists of—numerator statement. Discharges 3 months to 17 years, with ICD-9 CM principal diagnosis code of gastroenteritis or with secondary diagnosis code of gastroenteritis and a principal diagnosis code of dehydration. It says the time window can be determined by the user, but it is generally 1 year. The details I do not think you need. The denominator is population ages 3 months to 17 years in the metropolitan area or county. I guess it could be in the State as well.

Let me see if there are any exclusions you might be interested in. Numerator exclusions are pregnancy, childbirth, and transfer from another institution—equal to 90 days with any diagnosis code of GI abnormalities or bacterial gastroenteritis. There are no denominator exclusions. This is an avoidable hospitalization measure. It is not a provider-level measure. I mean the hospital provider-level measure. It could be the ambulatory care system.

Glenn Flores: I just want to comment. I have actually done some work on preventable hospitalizations. And the range of hospitalizations for dehydration and gastroenteritis considered actually preventable are 8 to 30 percent when you ask the inpatient attending primary care pediatrician and the parent. It is on the low end when you use more than one source. Actually, most gastroenteritis hospitalizations are not avoidable. I think we need to be careful about saying this should be a quality metric if in fact most of these are not avoidable.

Stephen Saunders: Are those data across the board or just looking at Medicaid specific? I would suggest that in Medicaid, it is probably a higher rate given the access issues.

Glenn Flores: This was largely, primarily the Medicaid covered population. It did include some uninsured, but very little in terms of privately insured.

Denise Dougherty: Your study was a qualitative review of—

Glenn Flores: No. This was a mixed method. We also did quantitative and multivariable analyses and looked at all the factors.

Charles Gallia: I had a hard time figuring out how to translate this into something that as a State I could do because it's population parameters that are general and then going back to the discharge data and then making that translation. The measure itself is great, and it has some value and utility. I think that in terms of being one that would be part of a core set, it was difficult for me to figure out how I would make adjustments in that population because of the catchment area versus enrollment.

Stephen Downs: This would be easier. You could envision doing it if your denominator was an enrollment denominator as opposed to a population denominator.

Charles Gallia: That is one way.

Stephen Downs: To Glenn's point, I was struck by the fact that you said the variation was from 8 to 30, which is a wide variation. Jack Wennberg suggests that somebody has excessive admissions.

Glenn Flores: No, it is not from hospital to hospital. This was if you took all three sources and asked them is this preventable. It was an 8 percent rate if you just said any one source. Actually, it was the PCP [primary care physician]. It was 30 percent. It depends on how you measure it. If you are taking a strict bar of all three sources, it is pretty low. It is 8 percent.

Denise Dougherty: One thing that maybe of interest is that the admission rate is much higher among 0 to 4 year olds, not surprisingly. That is four according to the submitter. It is 410 admissions per 100,000 population in that age group. For 5 to 9 year olds, it is 68 to 100,000 population, 10 to 14 is 31 to 100,000, and 15 to 17, 36 to 100,000 population.

Stephen Saunders: We were just discussing over here—the measure might make more sense if it was just the younger age groups because that is really where the issue is in terms of avoidable gastroenteritis admissions from dehydration. That is the group that has more morbidity. It is more at risk.

Denise Dougherty: To average it across 3 months to 17 year olds does not really make any sense.

Stephen Saunders: I think it would be a good measure for younger kids personally.

Charles Irwin: Are we ready to vote on this?

(Vote taken)

Denise Dougherty: Which one are we doing next?

Stephanie Kissam: (off mic)

Denise Dougherty: Is that an 85?

Stephanie Kissam: Yes.

Staff: This is the behavioral risk, health risk assessment for pregnant women.

Denise Dougherty: The short description is percentage of patients regardless of age who gave birth during a 12-month period seen at least once for prenatal care who received a behavioral health screening risk assessment that included the following screenings at the first prenatal visit: screening for depression, alcohol use, tobacco use, drug use, and intimate partner violence screening.

Participant: Is there a data source?

Denise Dougherty: A data source, let's see. Electronic medical record, it is a provider-level measure. To satisfactorily meet the numerator, all screening components must be performed. Let's see if there are any denominator exclusions. It is the first prenatal visit. It does not have to be in the first trimester. I do not see details about the screening tools.

Participant: Denise, if it is helpful, the numerator statement is right up on the screen. That is patients who received the following behavioral—

Denise Dougherty: We know that, but what Charles is asking is about how the first visit is defined, which I am not sure that is in here. I am not finding that these PDFs are searchable on my computer. If RTI could help find that measure on one of the submission documents, or you could call up the CPCF. I do not have that available.

Charles Gallia: Conceptually, it is a great measure, but I was concerned about the—this is not just an EHR measure frankly, because I am not sure that even in an EHR you would be able to identify the instruments that are there—the screening tool, or the notation methodology that might be available. It is potentially there. I do not know that it actually has been operationalized to know how representative it would be if you start drilling down into what is available. In some of the EHRs that we have looked at, these are text fields as opposed to something that may or may not be searchable.

Stephen Downs: I am looking at the notes that I took on this, and actually the application is very specific about using CCHIT [Certification Commission for Health Information Technology] certified coded terms. In fact, within the EHR, within a HITECH [Health Information Technology for Economic and Clinical Health Act] CCHIT approved HER, these would be coded terms and could all be extracted. The problem is that in the last AAP [American Academy of Pediatrics] survey, only about a third of pediatricians have an EHR let alone one that is CCHIT.

Participant: How about obstetricians?

Stephen Downs: Similar situation. I suspect it is at least as difficult.

Stephen Saunders: Doesn't this get at the debate we have had before about aspirational versus feasible? This is probably not feasible to collect now, but if everybody had an HER, it would be wonderful to collect because it is the right thing to do kind of thing. It is seems kind of—that discussion we have had on more than one occasion.

Charles Irwin: It seems to me that this is perfect to maybe be in the other category, perhaps in the payer group or in some other plans. This may be able to be operationalized better than in the Medicaid CHIP population. But maybe we could learn something from the other population too. We are not voting on that yet.

Marsha Lillie-Blanton: At this point, you all are putting forth your recommendations of what you think should be in the core set. CMS still could decide not to, but sometimes it helps to signal to us what you think needs to be there even if ultimately we decide it is not feasible because we cannot collect it. It is probably not likely we would add it, but I do think there is a lot of interest and concern, and we do need to identify measures such as this. That is why I said it is more the aspirational measure. It is your decision.

Denise Dougherty: And CMS can always disregard a no vote from this group as well because this was scored highly. It is not as if we are going back and dredging up something that got a score of 2 or something.

(Vote taken)

What is the next one you are going to show us? Radiation exposure. Children frequently undergo CT [computed tomography] scans. This does not really give us the specs we need. It does say this measure is part of a future measure hierarchy on health care associated exposure to ionizing radiation. Denominator: number of diagnostic CT exams performed at a facility. It is a facility level. The numerator is the number of diagnostic CT exams where metrics of radiation exposure are recorded in the electronic medical record. The data sources are administrative claims and the electronic medical record.

Clint Koenig: So, I guess if I am reading this correctly, where if a facility does not record metrics of radiation exposure, then it would not get counted.

Denise Dougherty: No. That is the metric.

Stephen Downs: The metric is whether or not you recorded the radiation—at the time that the scan was—

Clint Koenig: Right, but if I had a scan and inside the scan it does not say how many millirads or whatever was recorded then that does not get included in the numerator.

Denise Dougherty: You have a zero over something you get a zero grade.

Clint Koenig: This is to clarify, it is whether it is recorded or not.

Denise Dougherty: Exactly.

Stephen Downs: The goal of the metric seems to be to get people to record what the radiation dose was that occurred with each scan. It struck me as an interesting, but slightly misdirected goal. Reducing the number of scans might be a more direct effective goal, especially since there are not standard ways to report—at least not yet—standard ways to report radiation dosage.

Clint Koenig: Or is there significant variability from the way hospital one does a head CT scan from hospital two is just the absolute number valid enough. I move to vote.

Glenn Flores: Before we vote, Steve, if I remember reading this correctly that the point was that there are also exposure issues for children versus adults, and that you can regulate that if you are careful about it. There are a certain number of preventable cancers now that you can come up with based on prior research, based on the average exposure you have in the various procedures. I interpret it differently. And your interpretation would be very different from my interpretation in terms of how we would vote. I want to make sure that we are all clear about what it is that they are saying because they are different—

Stephen Downs:  I think my interpretation actually—I think we agree on this. The stretch goal here is to reduce radiation exposure to children because it does put them at risk of downstream consequences. What I was saying is that it was an interesting choice to target recording radiation exposure as opposed to targeting radiation exposure itself. It is clearly a step in that direction, but that is all I was saying.

Rajendu Srivastava: Steve, very aspirational I think, and there is a lot of new literature about this. I think there are cases if you do not record it, we are never going to make it better. But if people record it and are transparent about it and then you see this hospital uses three times the amount of radiation as this hospital, or what not. I am not suggesting that there are not problems with how this would all go about. It was interesting how they attacked it, I thought. It was particularly unique in a way. By including it in this CHIPRA measure set if it actually got there, imagine people were forced to do this. This would be transformational and how we think about quality.

(Comment off mic)

Glenn Flores: Karen, I totally understand it is voluntary. It is a question of there is a lot of clout behind at least saying it is in the measure set I think. That is all.

Participant: The denominator is an administrative claim and the numerator was whether there was a dose metric recorded. You will not get zero over zero. And then the other—I am trying to remember your comment.

Stephen Downs: My comment was based on the notion that it is not directly measuring the amount of radiation exposure. It is measuring whether or not that was recorded. That was comment number one. The other comment that I had made initially with it is that currently there is not a reporting mechanism for doing that that would not require an additional data collection step.

Participant: And I think you had a question about trying to cut down the number of scans. I think that gets into the question of it is beyond measure mobility right now to determine whether a scan has utility. It would be first to determine what the dose metrics that are recorded, and that is just the starting point.

Glenn Flores: Just going back to the measure, I found the passage about the fact that there are variations in the protocols, and a lot of kids are still imaged with adult protocols, which leads to exposures that are 10 to 20 times higher than what you usually use for children. I am really concerned by that. I would say this is a good start at least, just like Raj was arguing, that we need to start with are you even checking this because your kids if they are getting 20 times the recommended dosage, you are setting yourself up for a lot of future pediatric and adult cancers.

Clint Koenig: It sounds like what we are measuring is really variability and administration of radiation versus the scan appropriateness.

Charles Gallia: And to me the question was whether or not this should be included in the initial core set of measures that are reported by a State Medicaid program or CHIP. The specifications do not make payer delineations. My authority to do the assessments or even do the reviews on site and the facility. Not that I don't mean that it is something that should be again paid attention to, but having something listed in the core set doesn't translate necessarily to implementation.

Charles Irwin: Should we vote?

(Vote taken)

Denise Dougherty: This is kind of like the gastro admission rate measure. It is the children ages 2 to 17. Admission rate for asthma, for children ages 2 to 17 per 100,000 population. This is not a hospital quality measure. It is what is called a prevention measure or an area level measure of inappropriate hospitalization. Just one thing that has always bothered me about our area level measure is that AHRQ—it would not be a surprise to them that the underlying population is not the population of children with asthma, but the total number of children in that age group whether that is a deal breaker or not. The numerator is inpatient discharges ages 2 to 17 years with the principal diagnosis code of asthma excluding pregnancy, childbirth, transfer from another institution, age less than 2 years, any diagnosis code for cystic fibrosis and anomalies of the respiratory system. I do not need to give you the numerator details. Data sources are hospital administrative data. There are no denominator exclusions.

The rates are for 2 to 4 years old 279 per 100,000, 5 to 9, 165 per 100,000 population, 10 to 14, 76 per 100,000 population, and 15 to 17, 44 per 100,000. There is also a big racial disparity here, payer disparity, and income disparity.

Stephen Saunders: I have a question. I guess maybe I did not hear it. Is the denominator all Medicaid kids or all kids in general? What is the population we are talking about here?

Charles Gallia: It is a census-based population not anchored to the Medicaid enrollment or eligibility.

Stephen Saunders: Because when I read it, I thought it was the Medicaid population, which makes sense as a core measure. Asthma is a common problem.

Karen Llanos: I will put it in context for you. Just so you know, asthma—there is a different asthma measure. I am just putting it out there. It is the annual percentage of asthma patients with one or more asthma-related emergency room visits. Percentage of children ages 2 to 20 diagnosed with asthma during the measurement year, with one or more asthma-related ER visits.

Denise Dougherty: But is that a population-based measure?

Participant: No. It is enrollment—it is a Medicaid attributable one.

Glenn Flores: Just a couple of concerns. One of them is looking at the proportion of asthma hospitalizations that are preventable according to the work that our team did. It is probably about 53 to 82 percent that are not preventable, depending again on how many sources you use. And then the other issue of course, would be what is your baseline prevalence in the population you are talking about. As we know there are really substantial—I would almost use the wildly differing levels of asthma prevalence in the population, which also could be a major driver. And then on top of that, if the denominator is indeed all children, I think that really confounds the issue. This is another one I think that misses the mark.

Carole Stipelman: I am also concerned—viral exacerbators trump the asthma diagnosis. RSV, HMPV.

Participant: Any other questions or comments? Entertain a vote.

Kirsten Thomsen: I move to vote on measure number 12, asthma—

Stephen Downs: Second.

(Vote taken)

Charles Irwin: This measure will not be coming forward. Next asthma measure.

Denise Dougherty: This is submitted by NCQA, but not by the NCINQ [National Collaborative for Innovation in Quality Measurement] CoE program. The denominator is all health plan members 5 to 64 years of age during the measurement year who are identified as having persistent asthma.

Numerator 1 is the number of members who achieved PDC of at least 50 percent of their asthma controller medications during the treatment period, and numerator 2 is the number of members who achieved a PDC of at least 75 percent for their asthma controller medications during the treatment period where PDC is the proportion of days covered by at least one asthma controller medication prescription in the measurement year. Data sources are administrative claims, electronic health records, and other. Numerator exclusions: any members with any code to identify a diagnosis of emphysema, COPD [chronic obstructive pulmonary disease], cystic fibrosis, or acute respiratory failure.

Numerator 2, exclude any members who have no medications dispensed during the measurement year. Denominator exclusions: any members with any code to identify a diagnosis of emphysema, COPD, et cetera, and any members who had any medications dispensed during the measurement year.

The measure results are stratified by age to align with both clinical practice guidelines and reporting requirements. The age groups are 5 to 11, 12 to 18, 19 to 50, 51 to 64, and total. It has been tested at the health plan level and is currently being specified for electronic health records. Need anything else?

Glenn Flores: Denise, do you have information on whether this is based on pharmaceutical claims or how is this documented?

Denise Dougherty: It says administrative claims, electronic health records. This is medication management, percentage of days covered—detailed measured specs. It has a list of asthma medications and pharmacy codes.

Stephen Saunders: There is an existing NCQA HEDIS measure on asthma management that sounds very similar to that. It sounds like it is slightly more complex than the existing one. Is anyone here from NCQA?

Denise Dougherty: About 10 people. That would be the easiest way to get the information.

Participant: The existing HEDIS measure is appropriate is just looking to see that people are on medications. This one is looking to see that they were on for a specified period of time: either 50 percent of the treatment period or 75 percent of the treatment period. This one is looking at adherence. It is going a step beyond the existing measure. It is aligned to the existing one, but it goes a step further.

Participant: Is this actually a new HEDIS measure—

Participant: Yes. It is a new HEDIS measure. It was just added to HEDIS last year. It is completing its first year. It is in HEDIS now.

Participant:—retire the other one or are they both out there?

Participant: We have not retired the other one yet because this one is new. We want to give this one a chance to for health—start reporting this one and then later down the road, we could decide whether we would want to retire the first one or not. We were trying to raise the bar with this one.

Participant: But it comes from pharmacy claims data and then diagnostic claims data?

Participant: Yes.

Diane Rowley: How is the treatment period defined?

Participant: The treatment period is either 50 percent adherence or 75 percent. Is that what you are asking?

Diane Rowley: Treatment period to me implies a time frame.

Participant: In the measurement year.

Glenn Flores: I am sorry. I am still confused. Are you saying that is patient adherence or are you saying it is the provider prescribing the correct medication for somebody with persistent asthma? Those are very different measures.

Participant: It is patients. It is looking to see if patients were on the medication for a certain amount of time.

Stephen Downs: If you write a prescription, your patient does not bill the prescription. There is no claim for the prescription and it will not show up in the administrative data.

Participant: That is a function of it being an administrative measure. It is looking at prescriptions filled.

Participant: It is really a little bit of both because it is either you did not pick up the prescription or your physician did not prescribe it for you, which we know are both big issues.

Participant: Or you picked up the prescription and you did not use it.

Stephen Downs: This is a really tried and true measure. It has been going on for a long time. This is a slight variation on it. It has a fair amount of slippage. Deciding whether or not a patient has persistent asthma, ICD-10 I believe will let you do that, ICD-9 does not let you do that currently.

There is a little bit of slippage in whether you have accurately determined that somebody has persistent asthma, and there is a little bit of slippage, maybe more than a little bit of slippage in whether or not patients fill their prescriptions. But, again, you can always bump the number up by improving the quality of the care that is being delivered.

Like most of these measures, none of them precisely accurately measures what you want to measure. But if it correlates with what you want to measure, it becomes a useful measure.

Glenn Flores: I would say I agree with the nuances you are talking about, Steve, which is this is one of the most important things we can do for asthma patients and actually will probably impact the avoidable hospitalization rate of course. But as you say also because there is slippage, it is unclear about where the quality issue is. I guess in some sense it does not matter because obviously it is important that the providers are prescribing an effective medication for this group.

We know ineluctably that you have to do that. But the slippage is with whether or not the patient picks up the prescription, but of course that may also be in part due to the fact that the physician in the health care system have not educated the patient about the importance. Although I think there are some reservations about it, I would think that this is an important measure for us to think about. I do not know how you feel.

Stephen Downs: I actually agree with that because these measures are not entirely physician centric. They are health system centric. If there are barriers—we did some work where we found that a $5 copay led to about 50 percent decrease in prescription fillings. That is clearly a system problem. Measuring it at a system level would be valuable.

Denise Dougherty: It looks like there is a lot of room for improvement in the data that NCQA provided. Only 21 percent of Medicaid 5 to 11 year olds were at the greater than 75 percent PDC level.

Stephen Saunders: This is a major issue with Medicaid kids and not taking their controller—and so on. This measurement have been around a long time for health plans at least the original measure as I think almost every health plan examines this measure in the original format that NCQA has had it out for probably at least for the last ten years. I think this is an important one that we should consider for the core set.

Andrea Benin: How does this measure account for patients who are enrolled during the entire measurement period? Did they exclude it or they failed?

Denise Dougherty: Excluded. And that is why CHOP is working on some sort of better adjustment so that whole group of kids does not get excluded.

Stephen Downs: Motion to vote.

Elizabeth Anderson: Second.

(Vote taken)

Charles Irwin: All right. This one moves forward. I think we have three left. Tobacco use.

Participant: Can you tell us which other ones we have left?

Denise Dougherty: We have tobacco use—

Charles Irwin: And the Medicaid measures. We have three left.

Denise Dougherty: Tobacco. This measure is the percentage of adolescents 12 to 20 years of age during the measurement year for whom tobacco use status was documented and who received help with quitting if identified as a tobacco user. It is not just a screening measure. It is an actual follow-up measure. Recommended for health care provider level and population level reporting has been tested in the electronic health records. Specifications are provided for both provider and population level reporting.

The numerator is documentation that the adolescent is not a tobacco user or documentation that the adolescent is a tobacco user and any of the following. Advice given to quit smoking or tobacco use counseling on the benefits of quitting smoking or tobacco use using, for example, the 5A framework, assistance with or referral to external smoking or tobacco cessation sport program such as telephone counseling quit line or current enrollment in smoking or tobacco use cessation program. Numerator exclusions, none. Denominator statement. Adolescents who turn 12 through 20 years of age during the measurement year. Data sources are the paper medical record and the electronic medical record.

I am scrolling down for the results of testing in the electronic health record. This is a COE develop measure. It was specifically assigned as an adolescent well care and follow-up measure. It is for ambulatory, preventive health. Clinical guidelines cited. There actually is not a whole lot of solid evidence that this makes a difference though it certainly makes a different for adults.

Kim Elliott: Identifying the adolescents that are smokers is really going to be challenging. Electronic medical records are very limited at this point. You are not going to find it in your claims or administrative data at this point in time. It would be a very hard one for us to measure.

Denise Dougherty: This is for the provider to identify.

Kim Elliott: We still have to be able to measure it.

Stephen Downs: The point is this does not get recorded anywhere in administrative data so it requires a chart review. The second observation I would make about this is that all of the part for adolescents who are identified as smokers all of the criteria for counting those in the numerator, all of the treatment methods are all based on data on adult smokers. I am not an expert on adolescent smoking. I am sure our chair probably has some things to say about it. I do not think any of those have been demonstrated to be effective in the adolescent population.

Charles Irwin: Can I just ask a question? I forget what the definition of a smoker is in adolescents. Is there a definition here? I forget.

Denise Dougherty: If RTI could look for that, I will give you some results from the field test. Based on a manual review of the EHR, a total of 71 percent of adolescents had documentation of tobacco use status with a range of 58.5 percent to 94.9 percent across three participating sites. Overall, 14 percent were identified as tobacco users with a range across sites. Only one-third received help with quitting, range of 30 to 38.5 percent across sites, not very big. Interrater reliability was high for most data elements including documentation of smoking status. However, missing data prevented us from calculating kappa scores for all elements relevant to the measure.

There was moderate agreement between forms measured results and the automated EHR extract with substantial variation across sites. Stakeholder reviews showed the measure has face validity. We also found that the known groups' validity defined as the ability and the measure to meaningfully differentiate distinct groups varied by site. Performance rates varied by race and ethnicity and health insurance coverage.

Participant: Was there a specific question about this—there was a question about the definition.

Denise Dougherty: Yes, of a tobacco user.

Participant: Is any documentation in the record of tobacco use, it could be smoking or any other kind of tobacco, during the measurement timeframe? We did look at the different types of treatment. We specifically did not include medications for adolescents because there is no evidence about the use of the medications. But the other treatments we did find evidence to support them.

Denise Dougherty: And actually, the medications are not approved for adolescents.

Elizabeth Anderson: One of the things I was wondering about when I read this was whether the adolescent was a beginning smoker or—if you are 18, you could have been smoking for quite a long time and that struck me as—

Charles Irwin: There is actually some pretty good evidence that habituation occurs very early on really low dose. In fact, there is evidence now that you really have fMRI changes from very early doses. I think that is why it is good to include anyone who smokes at all in terms of interventions. We still do not really understand what the dimensions are within that group like you are saying. They have a good documentation here of what I think we should be doing. Collect the data or not. I am not certain.

Participant: Is the question about whether or not smoking status is documented in the chart? Yes. We found that it was documented. Now we did compare manual review of an electronic record to what we could extract from the electronic record. It was not great. It was a lot better than sexual activity, but it was not great. But the kappas were pretty good. We did recommend though as a manual review measure and will continue to look at EHR specifications.

One of the reasons that this measure was put forward is that if we do not start requiring the documentation and create the measure then the EHRs will not catch up. You can find tobacco status in the chart, but it is not always completed there and the tobacco status might be documented somewhere else. We did not have any trouble finding it and as you saw—I think Denise just read the rates. Only 30 percent of the kids that had it documented had any documentation of any kind of followup including just advice to quit.

Charles Irwin: I wanted to ask a question because I know that HEDIS has had the measure for substance use or alcohol use. At least the last time I looked at that was pretty low rates on endorsement with a HEDIS measure on alcohol use by providers documented in charts. In fact, we have done audits within our own system. In an adolescent medicine clinic, we have really low documentation. I guess what I am asking for—

Participant: Alcohol and drug use?

Charles Irwin: Alcohol use is a HEDIS measure reported. The provider documentation of alcohol use.

Participant: That is not a HEDIS measure. The HEDIS measure for—there is a HEDIS measure for initiating and engagement and substance use treatment for alcohol or for drug use. And that measure is a claims-based measure only. It looks to see whether people who have a diagnosis of substance use disorder whether they initiate treatment and whether they engage in treatment. There is not a measure—

Charles Irwin:—whether clinicians documented—asked an adolescent if he or she had used alcohol.

Participant: There is an identification of alcohol abuse, but there is nothing that is about use only. That is just the identification rate, which we do not even report. There is an initiation engagement measure NQF endorsed and it is used in some of the public reporting programs.

Alcohol is different. We looked at it. We considered alcohol and drug use measures as a screening for alcohol and drug use. We did not find sufficient evidence to support those. We did collect data on it. We found that it is much spottier.

Mary Evans: I think it is time to vote on this measure.

Charles Irwin: This measure we discussed at great length this morning and I am just wondering do people need more information again on this or do people feel that they know enough about this measure.

Participant: Let's vote.

Charles Irwin: Do people want to move this to vote? You have nominated it. Who nominated it?

Laura Pickler: I did. Let's vote.

(Vote taken)

Charles Irwin: This measure will be coming forward. This measure was also discussed this morning. Do people need more discussion about this? Have we worn you out so you will vote for anything now? It used to be that the measures would not get voted positively. Do people need more discussion on this measure at this point? Motion to vote. Let's vote on this measure.

(Vote taken)

Charles Irwin: This measure will be coming forward. Can we have a list of the measures? I think there may be five or six that came forward.

Denise Dougherty: I can tell you what they are from my list. It is HPV, the imaging measure, the asthma medication management measure, the coverage measure for all Medicaid and CHIP and duration for newborns.

Charles Irwin: So we have five measures that came forward. That means there were eight measures that did not come forward.

Denise Dougherty: That is right.

Charles Irwin: The question is can we put up those eight measures now again.

Carole Stipelman: I thought we voted out 79 on a strong medium weak and we did not get to do a yes/no vote on that.

Denise Dougherty: It did not get 70 percent on the strong vote in the first round.

Participant: Which one?

Denise Dougherty: 79 - the duration for all ages.

Charles Irwin: The direction for all ages did not come forward.

Denise Dougherty: It did not move forward.

Charles Irwin: It did not move forward. Less than 70 percent.

Denise Dougherty: That was the duration of first observed enrollment, which is not to say that is the end of the road for it. 056 is C-section. 55 is elective delivery. 14 is the newborn bloodstream infection. Healthy term newborn is number 58. Admissions for gastroenteritis, number 13. Admissions for asthma is 12. Number 90 is behavioral risk factors and prenatal care. 12 is admissions for asthma. Number 90 is tobacco use documentation and not initiation in treatment assistance.

Charles Irwin: What I would recommend is that we vote on each one of these whether we are going to bring it forward for the other because I think otherwise—I do not think we can vote for them as a group unless people want to vote for them as a group, but I think that probably will not work because there were some I recall, that people were less enthusiastic about. It is up to the group if we want to just put these as other measures.

Participant: (Comment off mic)

Charles Irwin: Can we do that? We will be doing power voting.

(Votes taken)

Charles Irwin: This one does not come forward. We will move on to elective delivery.

(Vote taken)

This one does not come forward. Next. Tough group here.

(Vote taken)

This one does not come forward. Healthy newborns.

(Vote taken)

This one does not come forward. Gastroenteritis.

(Vote taken)

This one does not come forward. The behavioral risk factors.

(Vote taken)

This one comes forward. Asthma.

(Vote taken)

No. And then the tobacco use.

(Vote taken)


Two of these of the pay to risk factors and tobacco use. We will go forward on five for improved core set and then the other. Do people want to also recommend the five that we have set out for the Medicaid group to also move forward?

Participant: Why don't we vote on that as a group.

Charles Irwin: Why don't we vote on that as a group. Do people want to bring the Medicaid measures forward also for the other programs for kids?

Andrea Benin: The measure that we talked a lot about with the radiation exposure. In the nomination form, the nominator writes that this measure has not been fully validated and field tested and everything. I did not bring it up earlier because it did not occur to me that the vote would go in the direction that it did. Are we just done with that discussion? Is it over? I feel like when the nominator wrote that it seems to me that is a red flag.

Karen Llanos: Thank you for commenting on that (off mic)

Charles Irwin: Can we choose to either recommend for five either yes or no to be included for the other programs?

Participant: I am going to move that we take the recommended initial core set and suggest that they be considered for other purposes.

Charles Irwin: Can we do one or two?

(Vote taken)

This goes forward? I really want to thank everyone for doing this. This has been a really complicated process. What I would like to do right now is in concurrence with Charles and Denise we would like everyone if they could take a bathroom break and if we could come back and for 15 minutes talk about the process and see if we could get input on how we might make the process, improve the process, make it less like root canals, and more like having a glass of wine. We will take 15 minutes and then we will let everyone go on their way.

Return to Contents
Proceed to Next Section

Page last reviewed March 2013
Internet Citation: 2012 Meeting of the Subcommittee on Quality Measures for Children's Healthcare: Transcript (continued). March 2013. Agency for Healthcare Research and Quality, Rockville, MD.


The information on this page is archived and provided for reference purposes only.


AHRQ Advancing Excellence in Health Care