Event ID: 2408663
Event Started: 7/25/2014 8:19:59 AM ET
Please stand by for real time captions.

[Indiscernible - multiple speakers]

Good morning everybody. We will get started. Welcome to the dark national advisory Council meeting. Thrilled to have you here in Rockville. We are coming to order at about 8:35 AM. My name is Bruce Siegel and I am chairing this group. I would like to welcome all of our NAC members. It does seem that our table have shrunk, maybe our room just got bigger. Something is change. I want to welcome our participants who will be joining us today in various roles. Those viewing us on the web.

Just as a matter of housekeeping before we do introductions. First, for those NAC members who need transportation after the meeting, please sign up at the registration desk behind me at the end of lunch. If you ordered lunch, please pay for and pick up the lunch in the registration area later on. For our audience in the room and on the web, if you would like to make a public, at 11:15 AM or 2:15 PM, sign up at the registration desk behind me.

With that said, what we will start with is introductions of our members here, as well as other people who are joining us. I know we have a couple of new folks who are with us today for the first time. I will start to my left here with Jamie and ask folks to go around the room introducing cells briefly. These years microphones so folks who are listening being can hear us as well.

I am Jamie Zimmerman, designated management official for national advisory Council.

Sharon Arnold, deputy director at AHRQ.

Henry [Name Indiscernible], Cleveland Ohio.

Sharon Davidson from national business group here in DC.

[Participant comment indiscernible - not within range of microphone]

[Name Indiscernible] pediatrician from group health Cooperative in Seattle.

Jane Crowley.

[Name Indiscernible]

Harry from Tufts University in Boston.

[Name Indiscernible], PDA -- pediatrician at CMS.

Sandra Becker from CDC national Center for health system fax sitting in for Charlie [Name Indiscernible] who sends his regards.

David Atkins, office of research and development FDA.

Jed Weissberg, senior fellow of clinical and economic review.

Carol [Name Indiscernible], national breast Cancer coalition. Patient advocate coalition in Washington DC.

Victor month Taurus -- [Name Indiscernible]

[Name Indiscernible]

Mike Johnson, chief clinical officer for [Name Indiscernible] health care in New Jersey.

Andrea [Name Indiscernible], chief medical officer for him are healthcare in Philadelphia.

Rick [Name Indiscernible], director of AHRQ. I add my welcome to Bruce and my thanks to all the NAC members for time and energy. I know how busy you all are . My colleagues and I greatly appreciate your work on our behalf and the good advice that I'm sure you will be giving us today. Also welcome [Name Indiscernible], I think it is your first meeting. To Sandy and to David Atkins from the VA.

Are we ready to go into the directors update, Bruce?

Before we do that, I would like to refer the draft minutes from April 4, which you all should have gotten ahead of time and have copies in your folders. Are there any requests for changes to the minutes?

Just a comment with relation to the discussion we had around patient engagement. As I read through the minutes, I know it is difficult to capture essence of discussion, but one of the things that struck us as we talked about three types of conversation. My sense was anybody who was not in the room did not or would not recognize that conversation was conversations with patients, family and the public. If there's some way to clarify that the minutes, I think it would be helpful.

We can make a revision. Jamie? We will make sure that gets done before it is finalized.

Other comments?

Is there a motion to approve the minutes with the edits that Michael just suggested?

So moved.

Second.

All those in favor?

I -- Aye So moved.

We have an exciting agenda. The first part of the morning we will do a directors update. I will start with a general update about activities at the agency since we last met, or at least a very small subset of activity since last met. I will talk with you a little bit about a project that we began and trying to figure out how to effectively pay for value. Spend some time on what we are doing with the portion of the patient's and outcomes research trust fund that comes to ARC. And then look for your help on ongoing work that we are trying to figure out about how to evaluate what we do at ARC . We spend a lot of time in evaluating other things in the healthcare system and have spent comparatively less effort on evaluation of our activities. And I would very much like On that.

Following this opening session, we will have an hour on delivery system reform work. You may remember that at the last meeting, Patrick Conway from innovation center and center for clinical standards and quality, talk to us about CMS efforts on delivery system reform. We want to follow that up with discussion of AHRQ efforts and get input on that.

In the afternoon, Karen [Name Indiscernible], who was from the office of national coordinator and health information tech knowledge he -- national coordinator for ONC will talk with us about [indiscernible]. And then we will have a session on AHRQ efforts on HIG. I am very much looking forward to your advice.

Updates first. Announcement of member transition. Jim Weissberg is retired from Kaiser and has join the Institute for clinical and economic review in Boston. Congratulations on that. Closer to home, you met briefly Sharon Arnold who joined as deputy director a couple of months ago, not very long ago. You see the big smile on my face. -- [laughter] -- this is but a very fine job from the beginning but an even more fine job now that Sharon, for such a young person, clinical and history, both in and outside government, she has worked at CMS in a variety of capacities. Ran the healthcare financing organization program, then in the private sector -- and I am quite delighted as are many of my colleagues -- I would say all except I have not canvassed them also might not be true -- [laughter] -- is joined is. Sharon, do want to say anything more?

I want to say that I very pleased to be at HRQ , I am still in a deep learning curve right now so I hope folks will bear with me but I look forward to working with the advisory committee members and continuing to help Rick move HRQ forward.

David Myers, who many of you will know, is joining the office of director as chief medical officer, which is a new position for the agency. David has been the director of CP three, center for primary care prevention and clinical partnerships -- that is not right. Director of CP3 . I am delighted that David will be joining office of director to provide advice from a whitecoat perspective on activities at HRQ. David is on vacation, a well-deserved vacation, so was not here tonight .

At the last meeting, that Bob Kaplan joint the office of director as chief science officer. Bob was not with us at the last meeting so I want to say that again and give Bob a chance to stand up and introduced himself.

Great to be here. I look forward to working with all of you.

The other major organizational announcement is we have created a new center in the agency. Center for evidence and practice improvement. This center is largely a consolidation of two centers that we had. Sent different outcome and evidence and CP3, the center whose name I just bundle. Primary care for prevention all clinical partnership.

The burden -- the purpose of consolidating these centers is to create a focus on practice and improvement. We had a lot of energy and CP3 around trying to improve primary care practice, energy and the center for outcome about how to improve practice and some decisions science work in COE as well. I thought it would be valuable to have the energy and work combined. There was quite a bit of interest in both centers around practice improvement, but note -- no focus within the agency there.

The other piece of rationale here is that COE managed the evidenced-based practice Center program, which is a big part Angkor part of the work of the agency. CP3 provided staff support to the US please -- preventive services task force. Much of that work is working with the evidence-based practice centers to produce systematic reviews for the task force to use. Again, I thought it would be valuable to have the folks doing that work to be more closely connected.

So we have combined the centers. We are currently doing a search for a permanent director. I would be delighted for input form any of you on suggestions there. We are looking for someone who walks on water, of course. A leader in practice improvement, ideally. Someone who has been involved in a major way in improving practice in the clinical setting. Also, ideally, someone who has knowledgeable about research. We are a research agency, producing evidence. So somebody who can understand the production of research. A leader and manager. And then also someone who can function in a government environment, which, as many of you know, is a wonderful challenge.

In the interim, David Myers has agreed to be acting director of the Center so David will be wearing two hats for a while. Chief medical officer and interim director hat. [Name Indiscernible], who has been the acting director of COE, will be the acting deputy director of the Center.

This is a very new -- I think we had a Federal Register notice published last week, although we have been working internally on this for a while. You can imagine there is a fair amount of process involved in getting a new center establish. Lots of energy as well is concerned that always comes when changes are afoot.

Scope and focus of the practice improvement side. It is still predominantly on primary care or brought into two surgical practice, coordination to the neighborhood?

We are certainly interested in the whole gamut. One of the main activities at this point in the center is the implementation of the accelerating P Corps initiative that David talked about last meeting and I will talk about again in a couple of slides. We are interested in practice and improvement much more broadly.

Just to build on the question. Are we talking about practice across disciplines? Looking at not just physician practice but nursing, therapy, social -- is that the hope in that?

We are certainly interested in that. One of the ongoing challenges of the agency is they are a relatively small agency with a potentially extraordinarily broad mission. Trying to figure out how to be effective in what we do is always a problem. Certainly of interest in disciplinary work, very broad approaches are needed.

Other questions about this?

Victor.

The evidence practice centers -- does this consolidation have any impact on that program?

Not directly.

The main eventual affect will I hope the, and even more effective use of those centers, as folks who were in COE and managed the centers directly, are more closely aligned with the work with the task force. And also worked a lot with the centers.

I should mention, I know that has been an amount of concern from the centers, we are we -- re-competing the contracts for the centers earlier than was originally expected. That has created concern. The reason for that is the contracts were originally less with the maximum dollar amount. In part because the task force has made greater use of the centers than was originally expect her's. And in part because other agencies here at CMS and other agencies have used more than was originally expected. We were going to reach that Earlier than originally expected. Because the work of the centers is extremely important, we wanted to the continue them so we were forced to re-compete and I know that has created concern. But that is kind of where we are.

Questions?

A very happy -- I want to recognize a couple of major awards that staff have received in the last few months. Bill Bain had a long career at AHRQ and is an infectious disease specialist. He received an HHS career achievement award, I do not know the number of people who receive this award every year, but it is not very many. It is awarded by Secretary. surveillance a few months ago and I want to congratulate Bill on that.

Sam [Name Indiscernible] who is an economist in the center for finance, access and cost run, shared the NIH CM foundation healthcare research or work for a paper that he co-authored with Tom buyer, Joe new house and a variety of other notables on integrating adjustment and premiums in healthcare payments. This is one of the very few awards that I'm aware of that had money attached to it. I think it was a $10,000 wars. There were seven co-authors on the paper, but congratulations to Sam on that.

I want to spend a few minutes now on -- telling you about some of the work that has come out of the agency since we last met. This would just be a subset of what we have been doing. I know that in some meetings we spent lots of time going through a very long list. I will not do that, but I do want to make sure you are aware of some of the work that has been published recently.

On July 3 the New England Journal published a study that was authored by John that freed late who was at the University of Washington in Seattle, reporting the results of a randomized trial of adding corticosteroids to anesthetics for patients suffering lumbar stenosis. The trial had about 400 patients and showed small benefits at three weeks after the injection for the patients receiving the corticosteroids. And really no discernible benefits six weeks postinjection. There are about 2.2 million Medicare beneficiaries a year who receive these injections. We have been in contact with colleagues at [Name Indiscernible], interesting to see going forward what happens with covered policy and with practiced policy around this very interesting study.

We just talked about evidence-based practice centers. In Southern California, a PC-based of the RAND Corporation published a report which is also been published in pediatrics updating work that the Institute of medicine debt in 2011 studying the safety of vaccines. That CBC report used the evidence that ILM used in the 2011 report and then reviewed additional 166 that is that have been published since the IOM did work. Similar to the 2011 report. That CPC report concluded that these are quite safe and did identify a few places where there were some areas of concern, but quite minor. Especially relative to the benefits. They also reviewed additional vaccines, including Emma cockle, rotavirus and a few others that were not included in the 2011 report.

Rick, could you remind us, is ACIP organized under CDC?

Yes.

I do not know.

Okay. In JAMA, David Baker and colleagues from Northwestern published a quite interesting study and valuable, I think, showing that colorectal cancer screening rates can be greatly improved among disadvantaged and vulnerable populations with targeted outreach programs called text messages, getting to people and saying you should be screened. The screening rates increased from 37% to 82%. As with a lot of the work that we fund, the challenge is now so what? And how you get this disseminated and into practice. David Baker and colleagues are working on that. We are trying to figure out and would be interested in any advice on how to move from this finding to real change in what goes on out there.

Jane.

What struck me about this is that it took six touches of each patient to get to that rate. So one thought would be to study or test if asking the patient what your preferred method at being reminded might allow the delivery system to be a little more selective. Because doing this for 1000 people is one thing, doing this for 300 million is quite another.

Six touches times five years for a test that is comparatively half as effective as colonoscopy. I am not up on the evidence. The burden on the health system to do these kinds of things and to do that with a very high reliability for no extra payment or in some cases no payment at all, is -- to me, that is a challenge. It is quite impressive. And it would be difficult to do it every day for every patient in every setting in every clinic in America.

A really good point. On the every day and every patient in every clinic, I think they need is clearly greatest in clinics that are serving vulnerable populations. Screening rates are already higher --

[Indiscernible - multiple speakers]

Still about 60%.

Which is that the a percentage of be but better than that 37% reasoning here.

Correct. And maybe others feel differently. It is impressive. Does anybody know all the top of the here and what the comparative effectiveness is for the single test versus colonoscopy?

Pretty close, I think.

Which we should be working on?

As a former gastroenterologist also, I can say about fecal chemical testing is that it is not a single test, it is a program of repeated testing. I do not recall in the article as to whether they looked at the success rate in getting people into a program of repeated testing or not versus once every eight or 10 year colonoscopy.

I believe it's have one see her for five years.

That is pretty good. The question if you do it every five years are you now primed to continue?

Exactly.

The thing that sparked what you justice -- what you just said is it reminded me of the reverse care rule. Which is imagination of patients that are more disadvantaged trying to swim in the deep end of pool and anybody dressed to jump in to help them out has to weigh much more effort is get them to stay afloat then people who were swimming in the shallow end of the ball. Everybody wants to jump in and help them out. We have places like Rochester, Minnesota, people without colons are getting colonoscopies and other places where it is much more difficult. And how will we make sure that we do not abandon those people who are trying to swim in the deep end of the pool just because it is hard to jump in and help them out. That is what sparked for me.

David.

I do not know the study, whether they use any automated systems, but to take this to scale, the assumption would be you would have to automate these or else incorporate to the extent that it is a live person, incorporate them into a broader outreach effort that when I just target for colon cancer but maybe diabetes and depression. We have wrestled with this and the VA as we have moved to a patient centered medical home. We have nurse care coordinators as a key component of that, but they are already being swamped by the responsibilities of what their outreach is meant to enable.

Certainly the text messaging can be automated and voice messaging can be automated.

Thank you.

I wonder if I could go back really quickly to the corticosteroid. In addition to the efficacy, and I've not read this paper, an ongoing issue we have in the health care system is contaminated steroids. Many of these are compounded. We have one of the largest [indiscernible] points for death in the country from this. Since that time, we have seen three more contaminations that come from pharmacies. I think is an area that is rich for investigation. I am still amazed at how little it is publicized. The state licensure over this is so very inadequate.

Thank you. Thank you for bringing that up.

If you could go back to the vaccine slide. I guess my question is whether there is any follow-up that is anticipated along the lines of practice improvement. Because, if this report is just put on the shelf and not used, then we may as well have not done it. I think the CDC published their 2013 vaccination rates yesterday in MMWR. I have not seen them but I can tell you that 2012 rates have a long way to go before they achieve the healthy people 20/20 targets. We really have a long way to go in this area. This is one of the main impediments are getting their, people's fear of autism.

Like I said, the CDC has main responsibility here. They will use this in their ongoing work. Do you have suggestions of activities that the agency should be engaged in to try to get this into practice?

Let me give that some thought.

The communication center has been engaged in messages on vaccine. California we are in the midst of addresses epidemic. -- Testis -- hard for people to swallow.

We have done some work for them our office of communication and knowledge transfer but I would not repent -- would not repent -- present. She is not getting your kids vaccinated and it drives me crazy. I shower this report but, I do not know how to deal with that and certainly the agency is not historically -- not been a big part of what we have done. I would be open to suggestions. Not clear what we should be doing.

I would be interested in Paul's take on this come about I can tell you the way pediatric practices handle this can make a big difference. Just to give you an anecdote. The pediatric practice that we use for my son does not accept patients if they do not believe in vaccines. There are all sorts of things that practices can do that I think can make a difference.

I agree that that is very controversial in pediatrics. At our foundation, we have been doing a lot of research on vaccine hesitancy. A lot of what we have found so far is that the most effective thing is. Pressure. -- Peer pressure. Much more than the pediatricians that many of these parents believe have been bought off by the big Pharma. It is mothers in playgroups saying I will not let you play with my children, so working to figure out how that [indiscernible] the peer groups we are heading.

I would just add a couple of things. I have a spouse who is a pediatrician and their practice does except families who do not necessarily believe in vaccination. The flip side of the coin, she had success in talking to these parents and convincing them. So there are two different ways that we can look at this. Also, just want to remind people, I do not know AHRQ is funded to be the biggest megaphone in federal dissemination. There are other megaphones you can take this message will further because they have much larger platforms for this.

In general, and I did not start with the mission slide because I've done it on the last 2 min. -- meetings, our mission is to produce evidence to make health care higher quality, safety, and to work with others to make sure that the evidence is understood and used. We are a resource to produce evidence. Mostly working with others to make sure it is understood and used. Although in a minute I will talk about a work in disseminating peak for where we have slightly different charge from Congress so it is not quite as clear is that.

Thank you for this very useful discussion. Henry, I'm so sorry.

As a pediatrician and thinking about all those who are involved in the care of children, including family doctors, nurse practitioners, physician assistants and other folks who are involved in the administration of vaccinations -- this area certainly is con traversal. The AP itself has policies and recommendations about how to work with families who are hesitant or resistant to receiving vaccinations. I agree with you, the charge of AHRQ is to develop the evidence . And I think our recommendation is that there needs to be some explicit strategy about health evidence that is disseminated to all those strategic partners and allies and stakeholders. I think that whatever the mechanism that is most reasonable, considering AHRQ limited resources, is something we suggest you take under consideration . Whether that is actually going to an AP meeting or send it out to another agency and they are responsible for doing that. Alternately, this research cannot die and hide within the organization. It will be ineffective otherwise.

Obviously, putting this out has limited in fact -- impact on a population that has been informed on the opposite population much more direct and charismatic way by stars. So we will always be at a disadvantage with dissemination. Discriminating people from how they believe will be a really bad idea. AHRQ has a history of promoting research and how to read -- work with patients in making difficult decisions. For instance, the [indiscernible] program that was funded by our funds a few years ago, that was intended to disseminate evidence in a patient centered way at point of care. There is a history direct that only HRQ can work, which is and health services research in patient centered care and connecting that with evidence dissemination. I do not think any of the other agencies are talking about that.

Thank you all. Very useful. The last of the published studies -- to more actually -- was a piece published in JAMA quite recently. This came from the Avery research. Deibert -- David might want to comment on this. This is an observational study comparing patients who added diabetes to metformin -- added insulin to metformin for patients with type II diabetes compared to patients who are simply taking metformin plus soften area. It should the patients are also taking insulin were at higher risk for cardiovascular disease and death. This is an observational study so it is suggestive of more work needed here to try to understand this better. Certainly created some concern I think. David, I do not know whether you are familiar with his work at all or have any comment.

I saw it when it came out but I do not actually know enough details. I think the question is can you adequately control for those selection?

They certainly did the best. Marginal for sure but at least it is provocative and encouraging for further work here.

The national healthcare quality reports and is asperity reports have been published since we last were together. These reports show that of the measures those are included there has been more progress on hospital-based measures. Particularly more progress that are publicly reported by CMS. So the measures that are publicly reported have improved at a much more rapid rate than measures that have not been publicly reported. A lot of the public reporting measures have been in the hospital area. Of the measures that were included in the report, recommended is [indiscernible] about 70% of the time and as in previous years, we document why disparities and access across socio-and economic and race. No surprise.

The last of the published studies that I have a slide on is a study that was published in J. It is a meta-analysis of earlier work showing the diagnostic errors on our not improve grant -- not improve grant -- in frequent and are a source of concern. The agency is planning to expand our safety work into -- it has been concentrated in the hospital setting. We are planning on expanding network outside of the hospital to nursing homes, particularly, ambulatory settings where a lot of the focus will be on diagnostic errors and trying to figure out how to measure them, and what to do about reducing them. This is clearly a broad area that will not get solved in 15 of but we hope to get work started on it.

A couple of other things to mention, not on the slides. We have released, a couple of weeks ago, results from the 2013 medical expenditure panel survey insurance components. So this is from a survey of about 30,000 employers around the country in which they are asked about their insurance offerings. A rich set of information about what they're offering, how much they are paying. Show their premiums in 2013 increased by about 3 1/2% on average from 2012. By historical standards, very low rate of increase.

One of our challenges the Kaiser family foundation does employer survey of about 3000 employers that gets released -- 2013 results were released in September -- 2013 results were released in September 2013. It showed a very similar result, about 3 1/2% increase. The survey that we do supports state-level estimates and is of value to people in states. Also with the 10 times larger sample, much more stable. So in 2013, we ended up with the same answers Kaiser. If you go back to 2010 and 2011, Kaiser had a lot of fluctuation in those two years for reasons that are completely not understood. The numbers were much more stable. But certainly coming up nine months later does great challenger. Still quite valuable information.

Lastly, on this part of the FDA, CMS published a couple of months ago a report that used data that we provided showing that adverse events in hospitals, a broad range of adverse events, declined by 9% from 2010 to 2012. This used information that relied primarily on information that folks here produced. One might ask why did this occur? The link to the partnership for patients efforts at CMS has been a very active effort working with most of the hospitals around the country through hospital engagement networks and a variety of other activities. A bunch of that activity builds on evidence that the agency hated first in the cusp programs focused initially on central line infections and more recently on catheter associated urinary tract infections and others. Substantial progress still a long way to go. Getting more adverse events than we should have. But progress there.

I just want to put a pen. We are and AGN contractor for the research foundation. This goes to an earlier conversation. We have witnessed in our own Hen really dramatic improvement in safety and reductions in readmissions. I would say that most of the intellectual content flowing through the partnership came through AHRQ in some fashion. So we talk about use and dissemination . Visible to the general public are even visible to some of us but an example of what is happening on an extraordinarily large scale in hospitals across the country. And AHRQ should be very proud of that. May not always be very visible, may have other agencies name or place on a -- we see it everyday. And having an impact.

Thank you, Bruce. I do see this as a model of poster child for how this should work. That we produce evidence and try to make sure it is understood and used. We have our colleagues from CMS buying trillions of dollars worth of healthcare, and very good set of folks to make sure it is understood and used. We are working with our colleagues around initiatives in nursing homes, particularly trying to make use of that [indiscernible] resources that QIO have and other work in hospitals. We have a long way to go to institutionalizes kind of relationship, but it was very successful.

I just wanted to add in the dissemination that this information is very helpful to me. We incorporated it into our evidence-based benefits of design committee, which has a lot of research and panel experts to make sure it gets incorporated into healthcare plan design. So if there are other things like this that you would like me to know about, I would appreciate it. We work in a diagnosis brief and right now we are working on compounding. You guys do all the stuff and we try to push it out to the large employers.

Thank you, Sherry.

Victor.

The agency struggles with recognition, particularly budget times in Congress and other things come at you to set something that is very important and Bruce has restated. A whole lot of HRQ inside. I wonder if there could be some sort of branding of that. As CMS programs grow, indicated that intellectual conscience is there. That might help.

Thank you, we are trying. Suggestion has been made, HRQ Intel inside , if we can get that out more broadly, I'm not sure. I should tell you that yesterday the Senate appropriations committee listed the bill that had they gone to markup, that they would have marked up. As some of you may have read, a couple of months ago lease, there was work done by the Senate Appropriations Committee on getting a labor HHS bill ready to be marked up. And then at the last minute, they pulled it back.

They posted it yesterday. It was quite favorable for the agency. $39 million increase over what the president's budget had requested. Four 2015. And 9 million more for 2014 and acted. Much of the increase for our safety work and HIT work but also increase in investigator research. Not to downplay the challenges, I appreciate your comments. But at least we also do have strong support in some quarters.

Back to the updates. Since the last meeting, the task force has received five draft recommendations for public comments. They are listed here. I will not go through them. The major changes in these draft recommendations from the previous recommendations are on the aspirin to prevent preeclampsia, the existing recommendations in I statement. The draft proposes is changing this to a B. And straining providing deficiency is not a topic that had previously been considered by the task force. The task force is proposing and I statement on that.

Also, since velocity, there have been six final recommendations released. Major changes from earlier recommendations. None of the finals were changed from -- finals were change from the draft but from existing recommendations on screening for cognitive impairment. This is the new topic that had not really been considered. By the task force.

Dental caries maintains the recommendation for use of foreign supplements and high-risk areas that adds new recommendations noting the effectiveness of fluoride provide in the primary care office. Hepatitis B screaming, the earlier recommendation was against screening and the population and the new recommendation recommend screening for individual of high risk with a grade of B.

On dissemination of task force recommendations, the guide to clinical preventive services has been published with an e-book potent -- coming soon and updated brochures for consumers that have been published as well.

Any questions on the updates? I will new -- I will move now for discussion.

DISSA,. We talk about how we disseminate some of this information. I think we can link this back to patient engagement. As was presented in the Secretary report, we talked about conversations that this is an opportunity. Engaging members of the patient family community. Who understand the concerns and needs of the family and the development of materials, presenting the evidence and a way that is understandable. Addresses likely fears among those. I think that is one area where informed patient advocate engagement would really work to benefit disseminating valuable evidence so that -- valuable information so that practice can be based on solid evidence [ Indiscernible - participant mumbling ]

Do you have suggestions on how we can effectively do that?

I think, just as this organization has adopted the SAC report which really emphasizes the patient engagement as a core of patient centered care, that mindset should be encouraged among all agencies that produce materials. That they should have regular involvement among patients on their communication boards or presenting evidence. It is a mindset that I think if we can help forward that as a research organization, that recognizes the value of informed patients input into the adoption of evidence, that might go a long way. Rather than doing and [indiscernible] or something.

I should mention -- [Indiscernible - multiple speakers]

I will think about it more but I think that -- I look at -- I look back at how patients got involved in the review peer -- peer review process and we are now. It takes time, but the value certainly has been recognized and I think we might be an organization that might be able to promote that in others.

I should mention, thank you again to Mike and Johnson and members of SNAC. One of the things I asked Bob Kaplan to do is how to effectively implement those recommendations. I will not put them on the spot and see if you figure anything out yet but we will get there.

Thank you. I know we will transition. I want to do one thing. I think we may have several members on the telephone who we neglected in third -- neglected to introduce. My apologies. I want to make sure there recognized. The for members who are on the telephone, if you are there, if you could introduce yourselves.

David Ballard, chief quality officer.

Thank you very much. Thrilled to have you part of this conversation. Please chime in moving on.

Mary [Name Indiscernible]?

Not here. Okay. David pension?

Nancy Miller at NIH?

I guess that. -- I guess not

This is Mary. I was on mute. Sorry.

Thank you, Mary.

I was out when the introductions came. Welcome. Thank you Ray much for being here.

Let us spend a few minutes now talking with you and getting your advice about a project that we started recently. Bob Kaplan has worked on this, Peggy McNamara and other folks. Larry [Name Indiscernible], who is with us today will be on the next panel is an IPA with the organization. He is a professor at wild Cornell in New York. Did a part-time IPA with us. Motive on this project.

As you all know that the mantra in healthcare financing is paying for value and natural and not for volume. The fee-for-service pays for things to get done but not necessarily the things that we really want, which is to get us healthier keeping us healthier when we earn -- R well. Tremendous interest in trying to figure out how to pay for value. As I think you all know very well, it is not so simple.. I will spend a minute on [indiscernible] has used quite effectively. In which he asks people to think about the difference and how we pay Apple pickers and federal judges. Apple pickers are paid for every Apple that is picked. It is the quintessential pay for performance. If they do not pick them, they do not get paid.

The measurement of what they are producing is tied directly to the payment amount. Federal judges have lifetime tenure and has salaries that are connected to any measured of performance or on the scale. But they are not based on what they do. So we asked why this is. Some people might say federal judges have a lot of political power, but burst response was this makes sense.

For Apple pickers, we can measure quite well without a lot of effort we want them to do, which is to pick apples. Might worry about about whether those apples are bruised. So I had to do something else there. It is not very hard to measure and we do not worry much about the measurement distorting the behavior.

For federal judges, what we want judges to do is produce justice. We are not very confident that we can measure well, how well they produce justice. We could perhaps rewards them for not having a long backlog or throughput of cases. Maybe that would be a good idea. We might worry that to the extent they have any ability to select which cases they choose, then they would select the cases that would be easier to get through the system. Or just rewarding for them not have a backlog that they would hear cases more quickly. But perhaps not do a better job of producing justice.

So we have this very week incentive system that is not connecting their pay during measure of their performance and we rely on professionalism and hope that they will do a good job.

I think this analogy or observation has some utility and thinking about how we should be paying for medical care. And what parts of it should have relatively strong incentive systems and which parts can we do a good job of measuring what we want the system to produce. And what parts may be more difficult. We are having strong -- where strong incentives make more trouble than it resolves.

There cruet -- clearly analogies here and work done on this for education. We are moving much more towards paying for performance. Concerns can be raised about teaching to the test and whether we can measure -- what we can measure is to what we want teachers and schools to produce.

So we are commissioning a set of papers to try to frame these issues into suggested research agenda that will produce evidence about how to pay for value more effectively. One of the challenges here is to produce a set of papers that will actually be useful to our colleagues at CMS and other payers who are tried to figure out how to pay for value. We have been in close contact with Patrick Conway and John Cavanaugh at others at CMS to in short that these papers will be useful.

Two more slides in this and then I will open it up for Russians and discussion.

Part of the goal here is to figure out for what areas of medical care should strong incentives be used. One thought might be that for a variety of elective surgeries, areas of care where there is more home to -- homogeneity, expected outcomes that are more appropriate for the care that is delivered right neurosurgeon for trauma victims. A lot Mark heterogeneity, measure what we want. Strong incentives might be less appropriate.

The variety of important questions about incentives -- use of incentives and how they might vary depending on the level of -- type of organization that is being paid. For making payments individual physicians as opposed to medical groups or hospitals or ACO, the answer to these questions to what extent are strong incentives appropriate, might well be different. As they might be different -- I started out with the fee-for-service payment model and a lot of discussion about paying for value, is overlaying not a fee-for-service model but increasingly we are trying to move toward a more bond of models. The answers to these questions may look different if we are talking about paying for value and paying for service versus paying for value in the context of global capitation. And the last bullet on this slide raises a number of issues that need to be dealt with in trying to figure out the answers to these questions. Particularly concern to the effects of Panther value on care delivered to people who are most vulnerable.

The last slide is the listing of the titles and authors of the papers that we have commissioned. The first paper is a paper about the theory of strong [indiscernible] and how the theory can be applied to medical care. Doug Conrad at University of Washington --[Indiscernible - multiple speakers] [Participant comment indiscernible - not within range of microphone]

There we go. The second paper summarizing what is known and not known about paper for performance and public reporting. And the effects on provider performance that was written by Martin Roland and Adam Dudley --[Indiscernible - multiple speakers]

Somebody on the telephone does not have the telephone on mute. Thank you.

[laughter]

I hope I did not scare you away, we are delighted to have you but -- [laughter]

A paper written by Mark/insurer at Yale and colleagues and the ways in which patient experience and patient reported outcomes can and should be used in trying to pay for value. And extremely important counterpart of trying to make this work.

A paper that Tom Rice at UCLA and a Bob have been working on. Four areas of medical care were strong incentives may not be so appropriate, what else do we do? The old system of putting money on the stump and saying here. Nobody wants that. We need some method of accountability. If there are areas where measuring value, measuring quality, will be problematic, what else can we do for accountability?

And then a paper that [Name Indiscernible] are working on outlining a research agenda. What we might do to actually get much more specific and helpful in terms of figuring out what areas of medical care are appropriately suit as Mike appropriately subject to strong incentives and what might not be.

These papers, we are expecting in the late fall, and look forward to a robust review and discussion.

These are great authors on key topics. Since you do not have to administer care like CMS does and many others, but this is a place where new ideas, for care delivery -- all my life, in risk invested outcomes, we have known that lack of good calibration of risk projections is a dangerous thing. If you under project, you will have incentives for hospitals to only take care the least sick. There so many of those examples. Those people who helped read those articles. It would sure be nice to have an article added there, and I do not know what the title or office would be, alternatives to this conundrum we have have for so long. Which is it does not work well. Although they are trying to do better, public reporting of poorly adjusted risk leads to all sorts of pernicious effects. I feel like we are in a moment of our care delivery that we should be thinking of completely different ways of handling the pay for value.

These are old ways that we have endured for years, trying to tweak. It would be neat if we had something new to do. Stomach -- semantic --

They are very well taken and how [indiscernible] deal with that at that and the paper. The pay for value, the last time -- I do not quite get, my senses this is new. We are in year five maybe of trying to pay for value and year two and a lot of area in year zero or minus and year two and a lot of area in year zero or -1 in a bunch of areas so this is all in development.

Payment for value, for supplies that you know how to do that. Risk adjustment is a key piece of that. I have written with those people for the last 25 or 30 years and we're all fussing about the same problem. Things have gotten better. There is better awareness. There was always that argument that the ways a perfect Internet thing or do get something going accept I wonder if there's a breakout idea that we could do that could be different? There are ideas out there. I think you might want to have one more paper there, which is really looking for the innovative approaches which are not the usual things. These are all adjustments of a long, wonderful tradition in health services research and care payment. Maybe we can do something new. They are not working. We are changing the care delivery system without a whole lot of understanding of what underling and so forth. Would we think of something completely different?

Richard, this is David Ballard. Stomach I had a comment.

Yes.

I think to [indiscernible] this'll pay for value, I think a very important issue is payments for value within delivery systems. What kind of incentives to accountable. Organizations have within those organizations? For example, I work for a not-for-profit, but about 50% my compensation relates to things like hospital what talent he, court measure performance, patient satisfaction. This is the case for many other large not-for-profit and for-profit systems across US. I think for this to be really relevant to the actual marketplace, in addition to how the federal government pay delivery systems, it would also be helpful if you explored what happens within those organizations in terms of incentives.

I agree with you completely, David. In this particular set of papers we are focusing on payment from other CMS or private payers to provider groups. The Christians you raise are extremely important. If you'll indulge me and wait 10 slides into the next session, I will talk about a project that gets very directly at the issues you raise. Sharon, did you want to respond?

Yes. I just want to say that as in our initial cut -- conversation we do have a lot of discussion about what we should be doing. Should we be doing marginal changes on the current path? Ores or something bigger new out there and can we write about that? We really struggled. I think that we have been pushing the authors to think about both near term as well as long term changes. The near term changes should be very practical changes that CMS another payers can use as they move along the current trajectory of their plan. But I really anticipate that all of these papers will have longer-term thoughts put out there. I think that in some of the summaries of these papers, there will be the kernel of what you're talking about and we can move forward with the research on those issues as well.

I want to go back one. I want to go back a couple of slides.

I think this is wonderful work. I might suggest that some thought be given to what the purposes of various incentives. Sometimes money is used to shine a light on something. I think the first CMS incentive program, HQ ID, certainly there was not enough money in it to change practice. It was used to show -- shine a light on [indiscernible] on arrival and other very important practical things in-hospital care. Very little money. But it worked. It worked phenomenally well.

Meaningful use incentives are an example of incentives that are used to support an investment or transition that needs to happen. EHR adoption is very slow, we believe it to be very effective clinical intervention delivery system change without money it would not happen. But it is temporary. It will not last forever. It turns into the [indiscernible] quite soon.

The third point, more relevant to your example about Apple picker versus judge, it is a change in the base payment method to better align it with the actual work expected. I think primary care is a good example. If we do not think now that the review method of paying primary care physicians works very well. It takes Paul and hour and a half to convince her mother to get the vaccine, that is really worth it. But in an RV you system, it will not happen. That is a permanent change and an inside about what is the work we are paying for. I think there is a fair amount of Martinus about -- amount of mightiness about the purpose of payment incentive. Clarity about that through research or otherwise might be quite helpful.

Good points. Thanks. V, Sandy, Andrea.

With a long career of participating in administering and observing different incentive approaches, my of this information is that people try to have very clear, reasonably simple standardized incentive programs as part of pay for value. When I observe the impact of and perhaps this can be in the first paper, there is heterogeneity of payment sent. People react very differently to standardize assistance, which is, gave. I think that is reality.

Thank you for making it more concave. I appreciate it. [laughter]

I have a negative reaction to this in that I have not bought into the notion that the only alternative, which is implied in your presentation, pay for volume is pay for value. I think there is a degree of consensus around pay for value has been too soon and too disseminated to suggest a different groups that about it. While he is experimenting with a, the top remains the quality and put efficiency at the top. I know the urgency of that and I understand that. But I also have a negative reaction not the notion of the main goal of the health care system will be efficiency, which is implied by the attention to pay for value.

I do not know if this is the issue, I think this is it good last with great authors, at some point we need to have an explicit discussion on what the impact is on the other quality domains of the overly focused attention on efficiency. What is a good debating center? What is the due to effectiveness? I know it is catching -- 20 capture that in the numerator, but at the end of the day, the attention will always be -- tends to be extremely sensitive to the denominator. Most of the attention is focused on war cost reduced. To my last point on this is don't you want to have some discussion about the bioethics of pay for value as you are discussing how to do it? And don't you want to have some discussion about the professionalism in that -- acting as an Apple picker basis. What does it do to professionalism?

Good points and also maybe I heard some language issues. Paper that time raise about Aaronson that will be very much concerned about impact on professionalism.. Doug Conrad's first theory paper. The language, paying for value. It does apply some ratio of output to input.

In, at least the language, explicit language in Washington, it is not so much a ratio. Talking about really more paying for quality. It is called paying for value. PQRS, trying to pay for quality on top of pay for services system. To Jane's earlier comments, what this looks like on top of a capitated system is somewhat different. It is not -- although it is called paying for value, the efficiency part, at least in the context of fee for services, not much there. Larry is one of the nation's expert on this. He knows way more than I do. Do you have any comments?

V, you may two points. Too much emphasis -- really talk about quality over cause. There's an effect on professionalism of Apple picker like incentives. I think the first comment, I agree with both of them, but in the actual discussions with the authors, I do not think there will be a heavy emphasis on cost. In fact, some people might argue there will be two little. It is more on quality.

The comment of incentives on professionalism, I think is a good one. As Rick mentioned, two of the papers and maybe three will address this to some extent. We could've commissioned a paper just on that but just limited resources. I think you'll probably be happy with what you see.

Sandy and then Andrea.

[Indiscernible - multiple speakers]

Call me next week. [laughter]

By topic, do you mean paying for value?

I want to go back.

Okay.

This is not my expertise within economics, but I am wondering if there's a missing industrial organization idea. Whether the unit of payment does not just concerned whether it is for patient or four episode or per case, but whether the optimal payment depends on the size and characteristics of the organization. If you are at a very small organization with a heterogeneous population, it is not perfect, that sounds a very scary situation to me.

There is a trend up and practices amongst physicians. Whether that is a good thing or bad thing, some deeper me think there is more risk adjustment, more efficiency, but how risk is red within the organization will effect the incentive. Even if the financial that you're paying is larger than the initial physician. I've always been interested in more work on that. I think it is really understudy. It is hard because how you pay depends on how organizations are structured and their size. On the other hand, the size of the organization, the characteristic, will affect how you want to pay. It seems to me that maybe that is one area that needs more research. I do not know if it is in this list. I do not see it explicitly --

It is not in the titles but we are asking the authors to directly deal with how the answers to these questions vary based on units that are being paid. And then David Allard said you should also pay attention to how the organizations are paying their physicians and other clinicians. That second question that David raced, we have not asked the authors to deal with in the set of papers. Trying to approach that through another project. We needed to put some boundaries on this.

I agree. Internal incentives are important. New organizations give financial incentives to their physicians or whoever. There is also nonfinancial. A lot of medical group leaders will tell you that if we do show -- and I think Kaiser has this -- if we just show our physicians there performance, and do not attach money to it and show them other people's performance, especially unblinded, they complain bitterly and say data is wrong. That is a very powerful motor for change. That is a possible area for AHRQ to do work on in the future. As Rick said, we had to draw boundaries.

I will read each of these papers theory detailed and thorough manner. I cannot wait to see them in the fall. That said, as everybody in the room has said, and knows, this train has left the station in managed care and in the commercial world. Employer groups -- plans on competing on how well they do pay for performance and value-based provider programs and what percent of their contracts of value-based. Similarly, with government programs, each state Medicaid department is asking the same questions.

I worry and what I think needs clarity is how we measure excess and what other metrics that we all follow to measure success. To your point, Victor, how do we make sure the quality is indeed the prominent improved factor here. It is not just a cost reduction. I think we have to put more money into developing those metrics and those performance metrics because the current set that we have is not good enough.

Thank you. [Name Indiscernible], David, Leon.

I just wanted to say that I completely agree with you about this idea of value versus quality and how we define value, especially looking at the different groups of consumers. Versus the physicians versus the payers. There has to be some clarity around it. I was a little bit confused, Victor, buyer, no deficiency. I would think, CMS standpoint, cost is very important because they are driving towards quality. When it comes to efficiency, we do not wait as much as we do some of the other domains we look at. We want practices, providers, facilities to be more efficient in their work but I think the bottom line is very patient centered.

Whether provider or clinic setting, there workflow is allowing to provide higher-quality care. That is what we are looking for. That is what will want to have. To make sure the patients are receiving the care when they need it. At a good cost and over high-quality. I also wanted to say that with the questions that you it mentions on the previous slide about what we should be focusing on, I think we might want to consider how you also pay for services that are provided across community organizations. The -- how would you pay for that system any care that is provided there. As a pediatrician, it is more obvious for us, we are thinking education, social services, you might want to think more globally how you pay for value in that construct.

I appreciate folks attention are what we are trying to do here. [laughter]

Very good comments. I think what I'm hearing and several other comments, unfortunately, the VAs a good case example of unintended consequences of paying for incentives so I hope that their newspapers there will be attention how to minimize the potential and unintended consequences of financial incentives even for quality. This is not about the denominator so maybe the title audit be changed about incentives for quality that sounds like it is living up a shared savings and other purchase and other incentive approaches to look at the true ratio. That is another set of cupcake conversations. Just focusing on the numerator, painful more. The third question is part of quality is owed of overuse, so there is a separate set of questions with choosing wisely about do incentives work differently if you're trying to get people to do less or something where there is good evidence that we're overusing it.

[ Captioner must disconnect line to transition with new captioner ]

Baiter, so it's maybe not the right word. It may come as a different one and when I looked at the list, maybe this is out of the --. In my role I met a juxtaposition of clinical care, education and the research model. So I would hope that some of these papers would address the impact of the incentives on our education system and the model for the trainees. And the challenge that the providers have providing clinical care, driven by incentives, how we balance that with education, and in the future education models for all of the training with other physicians or nurses, how you incorporate that in the learning model so they can be the next generation to help support what were trying to pick

--To do.

I think we will try to close this up and go to the last session. The discussion is been terrific.

2 points that I haven't heard I want to add to the conversation, one is about incentivizing and rewards on what you're measuring. I heard comments about that. Eight years of experience with the EQ I PEQI, one of the things I saw early on that focuses on the process versus the outcome. And I was fine with that because it was easier to make sure you're doing the right thing if you're incentivizing people --where are you incentivizing to get the outcome you want. As professional organizations, we need to figure out the right way to do it but the outcome should be the focus, looking at the outcome, the patient experience whatever it might be.

In addition to that if you're going to incentivize are look to have incentives drive behavior in this process, I would suggest we think about incentivizing collaboration.

So in taking the risk of doing a terrible injustice to your Apple --if the outcome is pretty valid, the apple pickers, orange pickers I agree true pickers, for incentivizing the fruit rather than the outcome, we will have a bushel of each of those sitting at the table. So how do we make sure cannot to compare any of this with an apple picker per se, but I did apple picking in my youth and it was per bushel. So I get that one.

But that to me is the process peace and it's easier to measure, but the real value is in the salad in this case. And to really take a riskier, if you think about I and the Apple, how do I fear --as the patient had wife well cared for by the apple picker. Trying to match those two things is the importance of focusing on the outcome and incentivizing collaboration. Some call it disciplinary, some call it collaboration, I think that is the crux were part of the crux of what we need to get out. Just one more thing to add to the pie. You're welcome.

Thank you for the suggestions and comments. We will share this with authors and be back with you with some results, probably not at the next meeting but hopefully by the meeting beyond.

And Patty.

Two things, when I what you get this right. I lost my son to to a position on a productivity contract. Unnecessary surgery. So this is an important thing, collaboration, it's huge. Because of second opinions, not doing it, so on and so forth. What I wanted to talk about, was back with your examination recommendations. And for for disclosure and transparency I am anti-brochures. And I'm wondering if we have any evidence on how well used brochures really are. We keep making them. And who really is using them? I see more gum stuck in them than anything. And they cost a lot.

Think about the AHRQ commercial, whatever you want to use for asking questions. Really funny, get people's attention. If there's any way we can start thinking more out-of-the-box instead of brochures, maybe we put these kinds of things in those type of --[no audio]

And that was my thought Kai of we could start thinking it that that spending dollars better elsewhere.

I will respond that Paul quick

What I said about unnecessary surgery, it's just part of the discussion, maybe from my perspective I was talking about the effect on the professionals, and moving people for apple picking. From where I sit, what we currently do is pay people for apple picking and were trying to move towards or figure out how to move more towards professionalism. I know I come from a system that tends not to reimburse people on productivity. But I think we are apple pickers. Were trying to figure out how to do that better

I think that's exactly right and Patty to your comment actually a good segue into both of the next two segments. Two things I want to discuss this morning. One is around what we are doing with the --trust fund that comes to the agency and we are charged with dissemination of peak or --how to know what effective dissemination would be part PECOR The next session to cover in this session , to get your help and advise on evaluating what we do, which is also what you are raising and challenging here.

So thank you for your comments on what is currently the paying for value project.

I want to spend a few minutes, or work with the PECOR trust fund . Once a lot of history here where we have been, compared to the --work. Starting with work well before 2005 with the effective healthcare program, titled in 2005. And we funded --in this program. Part of this funding, compared to the effectiveness review. I showed you earlier corticosteroids studies, the use of insulin on top of metformin particularly for diabetes, funding for treatment letter a versus treatment B, trying to figure out what worked.

We don't do that anymore. That is now the responsibility of the Cory, subsequent to the affordable care act.

We still have some work in that area but not very much. And if it's transom investigator initiated, traditionally funded if it gets I you scored, but it is not part of the program with the agency. So under very large investments, the development of registries --if you look at the people that have been funded by the Cory --a major effort from Cory to develop infrastructure information systems to support comparative research, most of the grantees under the PECOR net , feed money that started with money from the agency.

We invested as you all know, in systematic reviews conducted by the practice centers, this work does continue. It is part of our dissemination efforts, because to disseminate PECOR it is important that systematic reviews of PECOR. Not just one study resulting in the definitive information needed to change practice. So part of the dissemination effort is the systematic reviews and then we have invested in continue to invest in the Eisenberg said the --Center for clinical science, as Patty point out, that produces lots of brochures among other mechanisms of trying to impart information.

We did, under the large influx of research, to support this work. So this kind of history, on the next slide, is what we are doing now. What we are doing now is changed for two reasons. One most importantly, the affordable care act, which established PECOR and the different responsibilities than in the past. But also an important piece is we no longer have the Torah fund which supported a substantial part of our work.

As you all know, the affordable care act created the patient center outcomes research trust fund. And I get 16% of the trust fund to AHRQ , you can see on the slide when it asks us to do. To disseminate patient centered outcomes research to providers, patients, and papers. --Payers. The PECOR findings include findings that are funded by the court he as well is funded by the VA, funded by AHRQ , and the PECOR space as well as work funded by other private sources.

And they asked us to incorporate --information disseminated and to build capacity for PECOR through training . Two main responsibilities really, dissemination and training.

This slide shows the allocation of the trust fund with 80% of it going to the COR E for the content of PECOR. The COR E has interpreted that probably 16% to AHRQ for training and 14% to the office of the secretary to support the infrastructure so that PECOR can be better conducted . A fair amount of what the COR E is funded under the net effort to be considered as infrastructure work as well. The back this slide shows funds that have been received ramping up from eight received ramping up from 8 million received ramping up from 8,000,000 to 24,000,000 in 2012, --a fair amount of uncertainty about where we will receive in 2014 and 2015. There are 3 sources of trending --funding for the trust fund. And what was initially one dollar per person is now two dollars per person, paid by private employers, or by people covered by employer-sponsored insurance. It is just been difficult for the IRS in the treasury to estimate well how much that is going to produce. So we are expecting somewhere between 60 million and 100 million for 2014 and 2015, you can imagine the planning is a little difficult for this amount of uncertainty, given we are halfway through 2014.

This is a the portion coming to the trend seven, the 16 AHRQ, the peak or trust fund --PECOR trust fund .

And to tell you a couple of the initiatives that we are funding from the trust fund, and David Myers spent much more time on this at the last meeting. I want to briefly remind you, we issued an FOA asking for applicants who would work at disseminating PECOR to small and medium-sized practices, trying to figure out how to do this effectively, with a focus on improving performance on code here vascular risk factors. A large wrist --cardiovascular risk factors. And trying to figure out how to small and medium-size practices, what supports they need to effectively adopt PECOR into their Brack is and again --practice and again with a focus on cardiovascular --but more broadly to be able to adopt 1017 funding to implement as they emerge PECOR Sunday --funding to implement as they emerge.

Improving performance in these areas, we said we were interested in funding up to eight grantees for regional cooperatives, altogether potentially would include about 6000 physicians, probably taking care of 9 million patients. A significant dissemination effort in and of itself. Although 6000 physicians is a small part of the 200,000 primary care physicians we have in this country --and 330 million, so the evaluative work to figure out what is really working to make progress here for CMS and private payers, healthcare systems, armed with this knowledge would hopefully move forward and implement.

The applications were due on July 3. And we received a very robust response. There were 2 FOA is released here, one was for the update, regional collaborative, to actually work at implementing this, each of the grantees would be expected to do quite a bit of the value active activity themselves, and a second FOA for overarching evaluations. We also received a robust response for that second evaluation.

The FOA, and we will be going to the review process soon.

So we are pleased with the response there. And more recently we published at the end of June, an FOA looking to fund up to three centers of excellence with the health system performance in disseminating and implementing patient centered outcomes research. We are looking for applicants, for grantees, a cooperative agreement. We try to figure out as we look across health systems in this country, part of the question is what is a help system --health system --and what they have done to implement and disseminate PECOR in their practice.

And more broadly, to make progress in trying to understand how best to measure and compare system performance.

And David Bauer, you asked if we were to pay any attention to internal incentive structures, what is going on within systems. And we are asking in this FOA, --Fraser has been a motivator for send this, if you want to jump in and correct me if I mistake anything. We are asking the centers of excellence or grantees, to be paying attention to internal incentives and what are the characteristics of systems that are associated with different kinds of performance in disseminating and implementing PECOR, as well as other outcome measures for quality and resource use, what they are doing around --potentially broader questions about system performance.

We published this in June. The applications are due in mid-October, October 17. This is a bookend, parallel to the earlier FOA focusing on small and medium-size practices, and dissemination of PECOR there. Your we are looking at dissemination and implementation of PECOR in health systems.

I'm going to move to discussion of can't really to get your help and program evaluations. If there are any burning questions on the PECOR I will take them, but I want to spend a little time on the evaluations .

Thank you Rick. It is great to hear that these programs are moving forward. I was sort of struck by going from the previous conversation, doing the segue into this and your opening comment that Marco was dashed AHRQ was charged with dissemination . Imagine if you said AHRQ was it --improving outcomes in changing behaviors through dissemination. I think that's the mindset change we have been talking about in a previous conversation. Not just focusing on brochures. The generation of evidence with the examples you gave that is great with primary care practices, but you can apply that same thinking to the this is should about vaccines. Have you help pediatric practices and get them in this to show them what works, to engage patients and have those peer-to-peer conversations. That is really what the research has been. So thank you.

An excellent comment. I think the question for us is how do we make sure that the investments we make lead to improvements in the information that they shouldn't and physicians, other clinicians have one or making decisions and hopefully lead to better decisions. And with the PECOR initiative, very much directed at that . We have certainly heard from a number of folks and discussed internally that that initiative with the focus on improving performance in cattle Lou --cardiovascular factors, leaves out children. And we would very much look at future initiatives when resources are available that focus on pediatric care. A very good point.

And then, at least an introduction to a discussion to get your thoughts and advice about evaluation activities, and Patty, she raised this saying we put too much into brochures. We should be doing other things.

We have as I think you all know, a very broad set of activities in the agency. And the schematic, graphic was some of what we do. The accelerated PECOR initiative that we will discuss working on primary care, the dissemination of the PECOR primary care practices . With CMS funding, we have developed or are developing quality measures for the program. We have a national guideline clearinghouse with the national quality measure clearing house, we provide support to the US task force. We have partnerships with 47 states. And through that, we make data available through cost and utilization projects, information on and 97% of every hospitalization in the country. Connected to that have developed a quality indicators as well as a set of software that allows states and other folks to create public facing websites with information on quality. We developed and still provide lots of support for consumer assessments of health plans and now hospitals and home healthcare agencies come any other provider groups.

We provided support to practice research networks. And for many years have supported Certs, centers on therapeutics, centers that are doing research on how to more effectively use drugs primarily, there's a lot more work on HRT and other areas.

And we have a big advanced --investment in technology. We developed up the Department of Defense training program with 300,000 folks primarily in hospitals but now other settings hospital --how to better communicate with each other and work more effectively together. The collaboration issues. We have a program of research and medical liability. I'm not going to go through all of this. And I came to this agency 10 months ago, trying to figure out what should we be doing, where should we be making investments to most effectively use the resources we have can't to produce evidence and make sure it's understood and used. And almost, many of these activities have evaluations of the activity. A lot of what we do with investigator initiated grants, many of those grants are evaluations of something. But we have not had an evaluation of the agencies portfolios generally, or evaluations of some of the questions Patty has raised. Had we most effectively disseminate information. And we are working on moving forward there with the PECOR trust fund . But I was very interested in your advice on that.

Having said that, it is almost at the end of this session now. And I think the choices are --

[indiscernible-low volume]

I'll give you a little bit of time to think about it. And we will have some time to discuss it at the end of the day. We will take a 15 min. break and come back and discuss the efforts on the delivery system and performance. Take you very much.

[On a 15 minute break]

[Captioner standing by]

Let's get started so we are close to being on time.

So we have our next panel coming up. A distinguished panel. As we get back in our seats I will turn it back over to Dr. --to set the stage for us.

Thank you Dr. seal. I'm very glad to be able to spend time getting your advice about our work on delivery system reform. Paying for value, delivery system reform is among the Montrose of what everyone wants to do. I am joined, I will say a few words as intro, and then turn it over to Larry Castelli no and Irene Fraser. Larry and Irene are in your materials. Larry as I mentioned joined us --IBA, I don't know what it stands for. Intergovernmental personnel. Act. I was in the government a while back, it's used to bring academics into the government. And Larry's Casey's part-time. He is the Livingston professor of public health and achieve of the health policy economics and Department of Health care policy and research at Cornell. He is one of the nations experts and physician organizations and the effects of physician organizations on what positions do. As well as many other areas, and is a family physician by training for 20 years by practice. But no longer practicing.

[indiscernible-low volume]

And Irene Fraser is the director for the Center for delivery organizations and markets at a church you --at HRQ. Department of political science, done a lot of research on Medicaid , health care delivery, and has been at the agency for long enough that she among many other things, has shepherded --cost and utilization process, with main data access.

I just want to spend a couple of minutes reminding where we left off, Patrick Conway was at the last meeting. I will use a couple of his slides and then turn it over to Larry, who will talk about the research questions and one view of what we should be paying attention to. And then Irene will talk about some of what we have been doing. Then we will look for your advice.

So Patrick used this slide, the current state and future state. All of the things wrong with the current state of where we are trying to go in the future state. I added and took the liberty of adding the lifelong health system at the bottom on the right. It was certainly implied. We are trying to get their in figuring out how to do it.

This slide, which is pretty dense list of a dizzying array of payment innovations. The innovation Center is working on paying providers differently. Within the hypothesis that by paying providers differently, this will result in beneficial changes in healthcare and how it's delivered.

With a few exceptions, the initiative is not just paying providers differently, but for the most part it is changing the unit of payment, the terms of payment, and then as I said the hypothesis is hoped by changing how we pay, that there will be the beneficial changes that we showed, that shows on this slide, and how we get from Charon the left to the nirvana system of care on the right.

And this is the last of the slides I will use or talk from. He ended with how we might work together with AHRQ , and suggesting implementation of P-COR . It's crucial that we need developed science on core mission and improvement, transparency of data is important, training of scientists on how to do this work. It's important. And I have added the last bullet, which was not on Patrick's slide but we discussed it a little bit. I think an important part of what we should be doing it AHRQ is producing evidence about how to change delivery, the delivery systems how they can effectively change. So working with the delivery system, as many of you are interested in from your perspective, a new payment comes along and is supposed to change what you do but often the pressure is what should we change. How should we organize the delivery of care, what is the configuration providers, or to David's earlier point, the optimal use of internal incentives. What should we be measuring, had we get from a fairly blunt message of change to the payment system, to changing how we deliver care to produce the outcomes we want, we all want for our patients. That I think is --that it makes sense for us to be involved with.

Having said that, I will turn it over to Larry for further discussion of how we might get there. And Irene, of what we are working on doing.

So, I should say that we did not ask me to do the gives --disclaimer that I am just here --what I will put out is not agency policy. It is just some ideas, some of which I talked about with Rick and Irene and others, Bob and others which we haven't talked about yet.

Is that better. So I guess one thing that seems to me, the agency is doing now, is focusing more attention on the delivery system, delivery system research. And why would one bother to do that? I think most patients, legislators, the media, even most physicians they think health Claire --the healthcare delivery system is fine. And the affordable care act talking about the insurance side of things. That actually reading a lot about the one third of 2000 pages is about the delivery system reform, payment and organizational change pick

--Most people see a problem but I think what goes on the delivery system is very important, no matter happens with NIH, or whatever his are made, whatever clinical research shows, if the funds are not actually put into effect in the delivery system, it might as well not exist. And there are people who think that if that's true, spending so much money on basic science in the early-stage translational research and very little on the health delivery system to function better, is a mistake.

So that is a reason to try to focus more attention on the delivery system research. This is not showing up very well unfortunately. On my original slide it was good but I changed it to the that print format. Can people read this at all?

[indiscernible-low volume][laughter]

We will put it into a pamphlet.

It will be a widely distributed pamphlet. [laughter]

Or at least a leaflet. So in the middle line they will cross the medium structure process outcome. In the process here is considered more as a process for using nurse care managers to try to help patients with severe congestive heart failure for example, that kind of thing. Or some of the work CMS is already supporting with the --project.

Also added here, this wasn't a factor when it was originally created --the incentives that come from the outside are very important. That cultural leadership or it's important for probably how you get from structured processes. And for outcomes with patient engagement. This is an exceptional model in that you can focus on any of these arrows as areas for research most research has --. Most research has really focused on just today, evaluating incentives. --What is the effect of such and such pay for the performance program, or valuing be effective individual processes. There hasn't been that much of that work what is the effect of using the --nurse care managers at academic medical center say, to reduce readmissions for patients with congestive heart failure. That is great that limited generalizability. Isolating the one process part versus this model, is an optimal.

So the FOA that Richt talked about and Irene will talk about more, how the hell system performs compared to health system reforms to try and look at this more ballistically at least in my mind.

So can we learn more about what outcomes result from what types of incentives. There has been some research on that. How these outcomes are achieved. And not so much research on that. So what goes on within these organizations to make things better, and who, but how do, in terms of what organizations to achieve better outcomes, is it really true that the integrated healthcare systems achieve better outcomes than whoever, small practices. And there's a lot of assumptions made about this., There's not much research on this.

So I just missed and the comparative health performance --mentioned the comparative health performance, large amount of money that agencies put out as Rick mentioned. I think it should be very helpful, especially in looking at the how and the who. I think Irene will say more about that.

So just to throw out a few ideas, it is pretty arbitrary. Actually I want to go back, yes, it is funny, this didn't get published --. I did a white paper far --2011. [laughter] And so that outline in the systematic world, criteria for picking but could be --system research puts out a lot of potential questions. I'm arbitrarily picking some here --three years of the report. Just to mention a lot of others, I even mentioned --to Rick and Irene previously. Just to throw them out to give some sense of the kinds of things that I think could be useful. That are being done much so far.

One is, the FOA again will get to it somewhat, questions related to patients with outcomes research. Do delivery system reforms as they're happening now, with the CMS innovation center, trying to change the way care is paid for and therefore delivered, do they lead to more use of patient research outcome findings and what kinds of incentives does increase research leading to better outcomes. That might be randomized trials to show drug A is better than drug B. Are you getting better outcomes. What types of provider organizations are most likely to use patient outcome research and how can at best be disseminated.

I'm not generally to this slide but, I think the can probably be too much emphasis on the possible index of the delivery system changes and areas of care. The optimistic view is that the delivery system reforms could reduce disparities if you create --directly to reduce disparities for provide organizations. Or it just general quality improvement by provider organizations reduces disparities. General improvement really could have no effect on disparities or could reduce events, we don't know much about that at all right now.

Or the delivery system reforms that in --decrease disparities, we mentioned this in the earlier discussion, if it becomes a bad idea to financially take care of or for replication reasons, for public reporting reasons, the care for managed patients, there may be a tendency to avoid them. And obviously, if organizations are taking care of managed patients can do better, with performance measures, the rich will get richer and the poor will get forward. This is becoming very familiar.

There's not been much them. Or research on that to the delivery system. And some things there has been very little do this research on. Rapid an amazing demographic change in the US with physicians in all specialties cannot to private care, rapidly in moderate --employed by hospital, many --the specialties as well.

A none intended consequence I don't think, it is a been driven only by reforms that meaningful use and a lot of per ----have created an environment for physicians, we can keep going it alone. And they might actually want to be a large independent medical group, but there isn't a large independent medical group near them or they don't need them so they go work for hospitals.

And so, we know very little about the effect on quality cost of professionalism for that matter, possible employment of physicians. And the scary thing about this in my mind, it might be a bad thing. By the time we have evidence of what things are doing, this will be a phenomenon that will be very far advanced. And is probably irreversible.

--[indiscernible] by Stanford health systems in 1995, disengagement in 2001, year after I left and disappeared. The only practice in 2002. It's very hard for practice to go back from being independent after being with hospitals for a lot of reasons.

A quick comment on population health, this is very in vogue, what people usually mean by the patients that are attributed to our healthcare system. They learn --population health, that means what we are improving, the health of the attributed patients. That is great, to be an improvement on what we've had. Using that term loosely, and meaning attributed patients here --as opposed to the health of the patients in the community, the population of the community and up geographic area may lead to problems, less resources to the public health agencies, so for.

More to the distinction between these two conditions of population health and what one might want to do about it --cultural leadership, I spent a lot of time the last 20 years interviewing leave medical groups around the country and some hospitals as well. Especially the medical group, we will be asking them questions about their strategy in their structure. After a while this happens and we just laugh and say that the --the culture eats that for lunch every day you know what is a leadership very much because you want to --[indiscernible] it is pretty obvious that cultural and leadership are very important to be function of these organizations. And there has been I think in other industries a fair amount of attention to the cultural leadership but not much in healthcare. --Is done a little bit on physician groups but it's very hard to measure, hard to figure out what to do with his. More attention to culture and leadership, to the effects of performance the delivery systems, could be a good thing.

And this may be my last couple of slides. This is one, I don't know that Rick had time to look at these, if he had he would say what the hell -- [laughter] . I know he looked at it even late last night. This is the kind of thing we used to talk about in the antiwar movement. A long time ago. Organizing inapposite here. The first thing that strikes me was what's going on now is so many hospitals --starting a TOs where if you look at the situation objectively it doesn't make sense to them to do that at all. Certainly not in the next few years. If you asked the hospital CEOs why you're doing this they have reasons but it's the right thing to do, which of course it's always the right thing to do.

It makes sense a lot of times from a financial point of view. And I think to some extent it may be it's just that the spare. And the atmosphere, I think they will change this time so we better get ready for it. It may be correlated to everyone else is doing it so why should do it to., Doing a HCO .

In the case of one of the effects of the delivery system reforms, and with CMO's in CMS, it's creating the atmosphere where one might use a catchphrase, there might be a tipping point, that were just not going to do this now. I think that's important, it could mean that if the atmosphere shifts enough, this could become irreversible, and a good movement to doing things --even some of the earlier evaluations turn out to be not so favorable. I can easily imagine that HCO may not look so good. That anyway can you research into this, I think we might have practical relevance, that we could discuss I won't get into now. And I think that is the end of me.

All right, we are going to head directly into Irene's presentation and take questions afterwards. And Larry's last point is purely anecdotal but talking the hospital leaders a lot. --Putting various strategies. Things they are betting on, with the ACO, insurance projects getting into that business.

They don't know where it's headed in if they make those it might pay off. Some linkage of them.

I think there's three kinds of things going on. But that isn't so many anymore --and in New York Presbyterian, how we want to do this. It is very uncomfortable with the longer you're trying to generate volume as possible, on the other hand you're trying to do as little as possible. And that's kind of the position organizations are in now across the country. There are some that are all in and they're trying to move towards the value as quickly as possible. And I have to say, this is an optimistic thing, I have talked to the chief medical officer for a very prominent system in the South, that's just a financial machine. But they are actually moving very rapidly to get away from the traditional business model. And take as much risk based care as an value based care contracts as they can. Any say why are you doing this, and they say yes but first of all, we think things are moving this way. And secondly I've been doing this for 30 years and this is the first time in my life I can go to work every day and feel like everything I do is actually directed towards the right thing.

It does give people that opportunity to be optimistic.

I mean.

--Irene.

Okay, so taking off from where Larry left off, talking about systems and some of the research in the system. You probably can't read the little sign there, this is my example of a bad system in play. The sign says big-box ball bearing banana rollerskate and floor wax Inc. rollerskate and floor wax Inc. Several activities going on, each of which does a good job at its own thing. That when you put it together you can have about outcome. And I think whether you're talking at a small physical level like this, or 18 level, or at the hospital level or multi-organizational level, you can have the same sort of thing.

And so, and by its nature this kind of research is complicated, because there are so many factors that can play in and change the outcome.

So this is just a schematic of how I kind of picture research around systems and system redesign. You have of course the big black box and in this case a purple box, that people don't really totally understand. And it's often hidden. You have been sent is feeding into it, some information, about what kinds of models can most be successful. You can have an HRT which can help accelerate a move to a good model or accelerate dysfunction if misapplied.

And then it's important to be able to measure the metrics of whether that be the internal or the external entity to look at it. You need to be able to apply data to the metrics and be able to see whether you are in fact making improvements, or going in the wrong direction.

So AHRQ over the years, has focused on these variables in a fairly consistent, though I would say the volume of the resource as research has not been huge. There have been a couple of opportunities we actually had a solicitation on healthcare markets about 18 years ago. And the research around delivery system reform, using stimulus money and we did a variety of evaluations, and looked at dissemination and implementation, some other basic research, that was part of the effort that Larry was involved with. Kind of distilling from that kind of information, where we go next, with the gaps. There weren't any gaps --were many gaps.

In terms of the users of this information, the first 3, the competition, payment and the science of supporting, are things that for the most part to the extent we think about this been a take this evidence and run with it, the most part that is the payer or the policymaker. And the most recent effort that we have been involved with in collaboration with --is the science of public reporting, trying to get at the question we know providers pay attention but why don't consumers pay attention. Can they be designed in a way and used, can the data be approved, they can the display be approved in the systems and most importantly the consumer friendliness, the consumer centric this and paste centeredness of them, can it be improved.

And the last set, the delivery system design, are really questions that should be helpful, the answers should be helpful to those that are actually within the system. To actually help them with their performance.

You can provide all of the incentives you want, but if you don't also have some information about what kind of model is best to respond to those incentives, been that's not terribly helpful. Which actually gets at the point of discussion earlier this morning, about internal incentives. Those are really important if you look at the literature on incentives, what you find is that often they are ineffective because the people that are in charge of actually acting on them don't know they are there. Because there is a timeline. And just the operation, the translation of those incentives internally. And that the delivery system question, if you want change within the organization you have to start looking at internal as well is external ones, and looking at models that are successful.

That is one body of research that we have been doing, and looking forward to doing more on.

We've also been doing, actually probably in a more systematic way, we have looked at research focused on ways to actually intervene and make improvements. And I guess you could say this falls under the general rubric of dissemination. Dissemination is word that can be --what we've been attempting to do, is way to the action program, accelerating change and transformation in an organization. That works --what we have done is take bits of evidence of very promising interventions. And then piloted those to see if it works in this place, or if the can work someplace else, developing tools to try it further. A lot of the patient safety work that has been done, because and others --cusp and others.

One of the reasons that it's been effective is that it's led by providers and systems themselves and often rapid cycle, or at least government large organizations on a rapid cycle. Faster than traditional research.

The main sciences that this has drawn on is organization management and behavioral science. The effort is to get questions, not just didn't work somewhere but where does it work, how does it work, how long does it work and why didn't it work here or there, what were the factors. What does it take to make these kinds of changes sustainable.

As part of this there are generally tools, change models, practical guidance.

So I wanted to give you one small example of a recent example, this was a toolkit that used AHRQ quality indicators . And for the hospitals using quality improvement, and also used in public reporting, but these are much longer in quarterly --quality improvement. Creating a systematic way for those to be used in quality improvement. So we collaborated with hospitals to create this toolkit, and then to implement it through several hospitals. And as a result we were able to make very substantial changes in quality improvement. One hospital reduced stage III pressure ulcers by 66%, another hospital decreased 100 patients with complications of elective surgery by more than half.

And were working on a follow-on project to this. So that's sort of an example of the science of improvement. Which I think is important to realize the dissemination, alone, it's a concern in dissemination and implementation efforts. It is important.

And the final piece that we've been working on is getting this understand and use. And what we've done with that is partner ideally from the beginning, and certainly midway through, with the organization that's in the best position, or set of organizations in the best position to leverage change.

So for they Q I toolkit, for the science and public reporting, the amount that were making sure CMS will be getting all of the information from those that are in charge of hospital preparers. As we learn the best evidence, some of the states that are doing --reporting making sure they have the benefit of the toolkit we mentioned earlier, the web Biller --Web builder Monarch, so states that by default use the best science in the reporting.

Across the agency we have several really partnerships with --directors, for the overuse of psychotics in children. And partnering with nursing homes and QIOs on implementing strategies to reduce the incidence of pressure ulcers in nursing homes, with each trying to think about, welcoming your thoughts to about who are the partners we should be thinking about this. This research.

This is just a schematic of where we fit the the the the CMI and be a for example. Talking about design and piloting, design and tool building, and as we get into the implementation, and later into the larger scale implementation, that's where it's really cool that we do the handoff. And in some projects we take it further down the line to the large-scale implementation, and in others we can hand it off earlier, once it is clear that it works in all sites. It is important to have the evidence, not just the work in certain places, but we feel it will work in other places in the best time to do a handoff and when it will be going nationwide.

The partnership for patients is the best example of that kind of a handoff where things have been really right for implementation.

So that all kind of provides the context for the new initiative that has been mentioned, a couple of times. What we realized as we started thinking about how do you really disseminate any of it but in particular in this case, had you disseminate P-COR evidence in the not soft dissemination, rather than wait to see if it will work, how do you do that at a time when you have all of this rapid change very was talking about. So you can reach physicians in small and medium practices, and that is one of the P-COR projects, but in this massive scramble of systems, CEOs and everything, if you don't have the right CEOs, some have integrated systems but they're not all that integrated. You have this whole complex and rapidly changing set of systems. Had you actually implement and disseminate in a meaningful productive way, P-COR evidence.

So we decided to really tried to take out really big slice of this Apple rather than looking where a lot of the delivery system research is focused on, what do you do in room 27 A to improve things that we are trying to look at systemwide.

So what we are funding as we mentioned, three sets of excellence on the imperative health system performance. And the goal is to identify and possibly track, the pickup of P-COR and the performance of systems. And be able to look at the characteristics of the systems. So how to the characteristics of those systems affect the take-up in the performance.

And then taking into account all of those intervening variables that are so critical. The incentive system, the regulatory system, and --where there's been major consolidation, having different in the South than in the East. And getting all of those variables into account which is why doing it through these large center of excellence is making sense so, they are going to each have a substantial data core, and then the six research projects that come out of or use that core. And then we will operate with the coordinating Center that will be competing under a contract. And we will also have an advisory, technical advisory board etc., technical advisory board etc. So that's where we are on that. But then of course now that that is out the door, our next questions are what else should we be doing to try and answer some of these really critical questions that Larry mentioned. So I welcome your thoughts.

Thank you Irene and thank you Larry. Now we will go to questions from our group. As people start to put up their cards, I want to oppose what questions I really want to think about. You have a very strong anchor of medical direct heirs. One of the inform incentives that I think are quite large, not talked about a lot that happens more the state level were CMS is very involved, are what we call this the papers, delivery system incentive programs. These waivers, Medicaid waivers are now either approved or in place in the 11 states. Ranging from California to Massachusetts, Texas, Illinois has one pending. I think by the end of the year you will see the hints of dollars in Medicaid money at risk around quality, providing quality measures designed to reform incentive change in safe this health systems, high Medicaid systems, those that are most vulnerable. We don't really have a learning platform with the waivers. It's really sort of everybody doing their own thing to some extent, to be honest CMS is taking a strong hand in shaping these.

But nationally, there really isn't a learning platform. And also, each of these offering a prime network for dissemination. Some of them have actually formed their own within state networks on improvement whether trying to work around like Massachusetts.

I would keep an eye on those, and perhaps think about how AHRQ can be helpful there . These are under the radar screen. But they are very big and have really important implications. Just a thought.

[Captioner must disconnect line to transition to new captioner.]


[please hang up so new captioner can dial in thanks]

Sustainability and creation throughout an organization. And I think organizations themselves differ in how easily they do have. The ones that are the most fragmented -- you have something brilliant in the room 37 -- is the rest of the agency -- the organization remains unchanged. Did you want to add to that at all?

Yes, briefly, those are good questions. I don't have pretenses. I was glad that I reading set for today. -- Irene said but she said.

You have to distinguish between these efforts. For quality improvement efforts, it is extremely helpful for an organization to have access or be part of the work that AHRQ does -- where people learn together how to improve some specific things. You can then adapt that and AHRQ put up the information. You can then adapt that information for your organization.

That is valuable. That is a different question from giving sequential incentives for specific things. This is pretty much the way things are happening. And given the present organization in the healthcare system and the measurement and payment methods, went back to what you said earlier, that maybe the only thing we can do. But, really, ideally, we would like to have -- you would like to give an incentive for this and that. The ideal system would say here is the money. You take care of the patients the best for you think you should. We will measure some broad, important outcomes -- total cost -- whatever you think is important.

For some things, -- figure out the things you want to work on.

In any case, I think that collaborative processes -- strong evaluations -- the best ways to deliver with things it is not really the same thing.

Michael?

I want to say thank you -- about the presentations were great. To go to the question -- I would like to spend a minute on the culture and leadership PCU product, Larry. The fact that you typed in the structure and tied in culture and leadership as a way to move to the process whether they are hardwired are not -- the engagement -- I would say that the culture and leadership is a provider engagement. The reason it struck me -- as we have the conversation earlier about the value of your question -- quality over cost -- we are trying to mitigate to that speech therapist and dietitians in the organization of the last year. It fell flat early on. What seemed to catch people is what we added a multiplier. The formula now is value equals quality over cost times purpose. The reason you get excited to go to work everyday is culture, I think. For our organization is the mission and values. The idea of improving quality -- drive down cost -- being excited about it -- having a sense of purpose. This seems to resonate with the clinicians. This is the formula we are now using. It helps tie the folks in the organization back to the shared mission and values. I read the article in June by Tom and Toby -- engaging physicians in the healthcare revolution. This was a terrific article. What they talk about is helping positions along this continuum from -- collaborations -- they had four things he identified. Things that helped with that shift. Two of them -- one was a noble shared purpose. Another was the sense of tradition -- this goes back to culture. This goes back to leadership. The fact that you called it out here -- I am not discounting the other things -- but your point about population health as well made as well. I think that would be something that really -- because what we are talking about here is behavior change on a magnitude unknown to us up to this point. People are going to only change the behaviors of the people that are leaving them engaged them to do so. The whole idea of culture and engagement -- we really, really cannot overlook this. I appreciate this. Food for thought is a place that we don't want to miss.

If I could make a comment to -- Michael, you helped me be more specific in my thinking. Every day in my e-mail box I get things from the advisory board -- how to engage your physicians for hospitals. There is awareness in the industry. The world at large. Hospitals and physicians -- trying to figure this out. The AMA and the AHA -- for the first time in many years if ever they start to collaborate a little bit on trying to figure out some principles. But, there is really no research. Cultural leadership -- other things. Some value to looks at what seems to be the miles of position engagements once they -- that leads to the best results. This would be very valuable. When I say this, it is obvious. Until you said you said, I had not thought about that as specific research plan or question. Now, we might get some of that in [indiscernible]. It could be that some of the research project will look at that. If not, this is something we might want to think about. I think it is one of the most important subjects right now. Cultural leadership has a lot to do with it. The average physician feels like they are a pawn to the administrator. They feel like the administrator is a administrative waste. A prime example of administrative waste. Of course, the CEO -- they know what they think about the positions.

[laughter] A follow-up -- the observation of the shift to employ the physicians -- one thing we talk about and I think about a lot -- just because you get a paycheck and you are an employee, it goes back to your mindset. Do you work with that organization or for that organization to me, that is a very important difference. How do you help people think about with versus board -- with is leadership and for is apathy more often than not.

I will go to [indiscernible] next.

I thought it --

[captioner redialing - audio lost]

They wrote about this a couple of years ago. The implications on quality. Payment -- how we think about how you create a system that helps decrease the amount of adverse childhood events that would put them at additional risk later in life or individuals that have chronic conditions. Thinking across the lifespan how we want to pay for that and measure the success is really important. I appreciate you bringing that up.

2 questions that are based on what I don't know and hopefully, Larry, you are aware. On the can we learn slide -- you ask what, how, and who -- I was wondering on the What side -- are there any studies that have been done that can help help us think of examples on the kind of incentive the to certain outcomes?

The second question was on the hospitals that have physicians -- employing physicians in outside practice is whether there has been a study or look at potential of what the impact on the health care system might be.

There has been a fair amount of research into the effects of incentives on outcomes. One of the -- the second paper that AHRQ commissioned -- they will try. There has been some research periodically. This will be a fresh look and maybe more. They stand thoughtful and more innovative. By good people to look at that.

The evidence is mixed. So far. Positive and intended effects and negative and unintended effects. To my knowledge, I actually cannot think of any real research on the effects of hospital employment. There are all kinds of [indiscernible] --

Warren Baker at Stanford has a working paper -- he is trying to analyze that.

On cost, right?

Costs and I think admission rates, maybe.

That's right.

There are reasons to think that when hospitals employ physicians there are multiple mechanisms about that which could increase the total cost in the healthcare system.

Sherry is next.

Okay. I have a lot of things going to my head. I look at this differently. I am trying to figure out how to engage the commercial private payor employer world in this who are sort of wanting to move this forward and try to figure out how they can encourage these pay for value -- the top 10 services we should pay in bundled payments. How do we assess the good ACO's -- which ones they should get their patients involved with. How will this be different from the managed care the 90s that they had? There was a lot of backlash. What will the providers to differently for these patients? These are questions that I am looking at from a different perspective. What can we do to help engage them at get them to move this along? I don't know if that helps or not.

Just one comment on that -- one area in which AHRQ has tried to play a role is doing evaluations of private sector efforts whether it be a purported efforts or other efforts like that because we know from the CMS evaluation what is happening with Medicare and more recently Medicaid. What does don't tell us is what happens with private sector efforts which may be different. In particular what happens at the market level with the combination of what may be mixed or contrary incentives. For some think there may be a spillover affect. Those who gave the incentive -- if it is a beacon of one -- it usually is in. If there is a big enough incentive it may spill over to other payers. On the other hand there may be efforts to -- it may have a contradictory affect on the other payers. Some research that we've done using the data, for example, shows that areas of the country that are lowest in Medicare utilization and cost, at least for the senior population, may have higher costs for the private sector and this may not be an accident.

I'm glad to -- it's really not an AHRQ issue, it is more of a broad policy issue. There are commercial health plans that are going full speed ahead. Like Blue Cross Blue Shield of Massachusetts. To try to engage providers and mutually better for patients -- presumably ways to contact this. There are other striking every. We really want more for value we can get the health plans to do that. They still want us to pay is the same old way. The health plans is a the information systems won't support us doing things generally.

Now, if I were an employer or coalition and I heard them say that, I would say wait a second, why are we paying you? Your claim to fame is that you have great information systems and now you're thing you can do the right thing because your information -- you can do it. I don't want to hear that. So, I would like to see a lot more of that from the employers.

To build on that, one other point is that in a lot of these markets you've got the plans and providers in the middle because they are being paid half one way within incentive to do something and have another way with an incentive to do something else. So, how do they struggle that? To they just tried to jump over to something? To they strike to stay with something? In many cases they don't have a choice if they want to get the market share. It makes for -- a researcher's dream in many ways had a nightmare and many others.

Sandra, Hendry, [indiscernible], and Jane.

The first question that comes to me what I think about the delivery system reform is can we describe the delivery system? Can I do it accurately is in each state or locality? I honestly wouldn't even know what to say about that.

Even the trend toward increasing ownership of physician practices by hospital, I don't think is uniformly documented. Well-documented. I know that [indiscernible] is involved in a never -- that's probably the best there is. I will have to look at what warranted. It is difficult. We surveyed physicians but a lot of time they don't know everything about the organization. If it is a large organization --, the people would have. So, I don't know if there is an innovative large thing you could do like [indiscernible] -- they are partnering with state to describe the system. Rick knows this -- I thought what Andy [last name indiscernible] does -- they have a partnership with the California medical board. They sneak in some questions to the physicians about the practice. In the process of relicensing. I don't know if there would be a possibility to think about this -- you could do it partnering with state licensing boards for positions in other types of providers to try to provide -- the physician cannot answer everything although he or she might know of someone that could. I thought that HCUP was such a successful effort -- I thought the partnership was interesting. I did know if there was anything that could be done to describe the system that way. It obviously would be a large effort.

Briefly -- certainly the things that we talked about here -- part of what we are looking for -- grantees -- the description issue -- under this issue, we have developed a survey instrument to try to get better information about physician organizations and in collaboration with NCHS -- this is being piloted now. We agree that getting the descriptions right is important and part of what we will try to figure out how to do.

Thank you. Next up -- Henry?

I have to think both Michael and center for setting up my [indiscernible]. I don't know if you read each other's minds about this -- the thought about -- this has to do with the faculty development hat on now. As I became recently the interim director and the interim chair for the department, the issue of physician engagement -- disengagement -- this is a huge issue for the 830,000 active physicians in the US -- the 2012 count from the Kaiser foundation. One of the issues and challenges that the physicians have is where we have been participating in the maintenance of certification. I think this is a great opportunity -- a level of granularity much deeper for the AHRQ in many agencies. The part of improvement projects where most positions are required for the moment and there is a lot of discussion between the AMA and the Board of medical specialties -- whether or not this will be a continued process. What is required right now and ongoing for all of us that are active physicians to engage in a micro practice level elegy improvement project. Is there an outcome of goodness? The question is, ultimately, what is the effect -- the sum of all the projects? This is the fundamental question that the AMA and many other critics of the [indiscernible] -- the way it is scheduled a graded now -- there is no evidence to say that it is effective.

I do believe it is a good thing. The question is, how good?, Trevor should be placed in this? The onus is now -- for most of these positions -- we are still working in the revenue-generating world. We haven't gotten to this great equation purpose. I want to get there.

I think it is amazing. I will beg, borrow, and steal and put this on a slide in the future. Until we get there, the reality is that this is how physicians work and how they train and this is how they maintain their certification. There may be an opportunity to become involved as additional stakeholders in the private sector. Frankly, the physicians also need some advocacy. AHRQ would have a great role in that.

I will make a comment on that if you want to respond to a -- I think what Henry said is really important. This is the way that most American physicians and counter quality improvement.

Yes.

We don't know how it works and if it works --

Absolutely.

My guess is like anything else -- anything they can be done well can be done badly.

And evaluation would probably show this.

Thank you, Henry. Next -- no will. -- Noel And then Jane.

This is exciting. I have a couple of comments. First, Larry, I know this was unintentional, but you skipped over the disparity slide. I hope that disparity is incorporated in all of your thinking about this and I hope that disparity is not be prioritized by AHRQ. It is really important. The second point that I have is -- this is really about where to place your bets. This is the right question. I don't have the answers to the question, but a couple of thoughts. One is whether you want to invest in tweaking the margins or whether you want to invest in a blank slate or some sort of mix of those two things.

I know there are have been all sorts of experiments out there already. Before I place the bets, I would want to summarize the experiments and what the evidence suggests so far. I will give you one example that has been meaningful to me. The Atlantic City experiment that [indiscernible] did along with [indiscernible] Epstein and some other people.

While they had great results, I'm not sure that the results can be generalized to other settings in the US. There is one component of what they did that I think is important. I didn't see this on Larry's slides and ideas. This has to do with team science. It asked get back to what Michael was talking about -- collaboration.

I know we talk about picking apples -- I never did that as a child. I want to use a sports analogy about team science. I don't know if any of you saw the David Brooks commentary in the New York Times two weeks ago about whether your life is more like baseball or soccer. I take most of us -- the point was that most of us think our lives are more like baseball. But, when you think about it, baseball is really a bunch of individual contributors. If you have a couple of good pitchers and a couple of good hitters, you're doing pretty well. But, life is really more like soccer where there are spatial relationships. Everyone has to respect their position. If you applied that analogy to healthcare, I think it is important to get our minds around how we work as teams and how we collaborate.

Thank you, Newell.

Jane is next.

I appreciate this conversation. I have two comments -- I was struck by the slide and subsequent comments that most people do not believe in the case for change. Or that they are satisfied with the status quo. That is not at all my experience. The employers, physicians, community members are pretty unhappy with the results of our healthcare system. It does suggest that perhaps a case for integration -- a phrase I would use to describe this delivery system reform -- is not strong enough. It's just like the vaccine example -- the public must not believe it. The early feedback from the first year of high-value networks is that they are too narrow. Give us more choice. So, we are perilously close to going back to the 90s where open access was the seat as quality. -- Perceived as quality.

It must be higher-quality if I could choose to go to the orthopedist. I would suggest that if we go bear the position that believes that steroid injections work, perhaps because he or she gets paid for each and every one of them, what the evidence is there, will belong. We will just get more of we had if we ate at the first bit of resistance which sort of looks like we're about to do. The case for integration must not be strong enough. It has died down with EHR, but people like I and others have said we have to have electronic records. Articles doesn't work and it doesn't make a change. Thank God we did it anyway. The evidence was mixed for a while about whether EHR would change anything. That was in the hospital -- most of those studies were inpatient. Never mind the potential for improving care because EHR's are now on bigger platforms. It strikes me that there must be visibly verse -- disbelievers that integration is necessary. Group health and a very few others of the plans that consistently produce five stars. This is the established way of measuring population health today. There are no more integrated models out there, right? They are the poster child of integration. I feel like what are we waiting for? The evidence is here. Perhaps we need to disseminate it more effectively. Integration is required to get to quality.

We are not going to get to -- my opinion -- elimination of disparities and improve safety unless we have an integrated health system in this country or integrated health systems -- the plural.

The reason that [indiscernible] worked -- right? How effective can ascension be if we did quality improvement and each individual hospital and nursing unit at a time. Mercy health or any of the largest is a -- we are doing it collectively and learning. I am frustrated because the case for integration is apparently not very strong.

I think it is a terrific research question. Dissemination question. I think it is pretty clear -- I am hopeful that we will be able to get evidence out of this new okay on that. -- This new FOA. We know there is a set of integrated systems -- if the market forces others to anyway, do they get five stars, to?

[indiscernible] Yes, that is an important empirical question.

The other comment I was going to make -- Irene said that almost nowhere is anyone at 50-50. You are either totally [indiscernible] or most of us are in the 5 to 7% range. The variable is very confounding in day-to-day life.

Just a comment on that. In terms of integration, for some people integration is viewed as consolidation going too far.

Yes.

That's fair.

A lot of this will be decided in the courts as well.

The ninth circuit -- around the Idaho issue -- involved with this. Integration is great, but the law doesn't allow to the extent you done it. Sorry. I think we have a ways to go yet.

Jane, a distinction between clinical integration and financial integration can be large. As Larry pointed out, the physicians don't know much yet about that and it doesn't lead to the kind of clinical integration you are talking about.

Yes.

The narrow networks -- for the most part, as far as I'm aware, this is nothing to do with integration. They could, potentially. But I don't take, at least in the areas I am familiar with, indicating that the providers and clinicians are actually any more integrated. They might eventually get there. They might have a different understanding of the networks.

The department of justice to the mission is pretty demanding. That's what I mean when I use the phrase. Perhaps they are ahead of the clinical agencies in saying this is what you have to do. They came at it from the perspective that you can't split the pot until you approve the [indiscernible]. It is a demanding task.

Last question is to have a.

-- Harry.

In full disclosure, my research is in making risk adjusted models providing decision support built into health information technology to do the kind of things we are talking about.

I have it, to make. Listening to all of this, as we pick about applying these things in large systems we start to think about economists and we think about systems and policy. As well as clinicians. That is excellent. This has been good comment. Besides being a researcher, I am also on call and I see patients. I hear stories from patients that are hair-raising. Places where we have good quality improvement programs such as the hospital I am in. I hear horrifying things. In our last meeting we talked about involving more stakeholders than patients experiences. In this improvement of the efforts, we can't just use the metrics. I am not a qualitative researcher, but I think we need to learn how to get the inside -- things we would otherwise not find from the experiences of the patient that even the physicians in their care. Measured in a different way than we are doing it quantitatively. There is a lot that will be lost otherwise, I think.

Larry?

I agree with him. This is all go -- also -- I know a lot of patience -- no matter how bad their experiences, they don't necessarily get that this is a systematic problem. I had some bad experience is myself. And France. I have to remember -- it is not just a one-off that I had bad luck. One of the things that we commissioned is about the patient experience. We have some. People doing this. The question we posed to them -- can patient experienced the used in new ways that [indiscernible] -- Is good -- but to provide more global measures about how this is, really.

I think this is an important question. If you try to measure integration by claims or measure it by provider surveys or by whatever, or coordination -- is tough to do. But, if you can get some answers from the patient's about things that relate to their coordinated care, you can learn a lot. I think this is an important point. I am optimistic about what we will get out of this area

Thank you all very much. I appreciate the panelists being here. Very much. We now must move on to public comment. No one has signed up for this. Is anyone in the room that wishes to comment? Hearing on -- I know some members are on the phone. Do they have any questions or something they want to say? I don't know if they are still there.

Hearing not, it is time to break for lunch. We will break. For the members here, you can buy lunch -- behind me in the registration area. We will reconvene at one o'clock. Thank you.

[The AHRQ meeting is on a lunch break and will reconvene at 1:00 Eastern Time. Captioner standing by]

[Captioner must disconnect line to transition to new captioner.] [Captioners transitioning]

[Captioner standing by]

Okay, we are going to get ourselves going again. If we can come back to our seats, we have a special guest who has other commitments today. [indiscernible]

All right, our director now to make introductions.

IQ Bruce. I'm really delighted that Karen Salvo, the national coordinator for health information technology, is joining us this afternoon. Karen, a 20 year career focused on improving access to high-quality care. She has been at the department for four months? Six months. Time flies. She's catching up to me. Before coming to the department, was the Commissioner of health in the city of New Orleans. Spent many years in New Orleans trying to make healthcare better for low income folks. We had a very energetic morning. I'm sure we will have a more energetic next 45 min. next 45 min.

Thank you Rick, thank you for the chance to come speak with you and share where we are going at ONC with health IT. I will try not to speak and letters. I want to thank AHRQ for helping shape my career .

I had the luxury of knowing John Eisenberg, and seeing Carolyn Clancy speak standing in for him in a meeting in New Orleans. And that's what I want to do, to understand how data can help decision-making. --I got to know her over the course of the year --and so I am thankful for the mentor show --ship and role modeling, the support I've been able to do in my career.

I want to share with you all today some level setting about ONC, where we have been is a country in health information technology and where we are today, some of the challenges we are facing, and what ONC is doing to address those in partnership with the private sector and you will. And hopefully a chance for a lot of questions. I will try to keep myself to no more than 20 min., maybe 15 min. given the time of talk.

ONC was formed in 2004 by an executive order of George Bush. Dave Prater was the first coordinator. They started this agency at a time when there was a new idea that we could digitize health information. In a very aggressive and frogs Bactrim way across the country, so it affected the country using electronic information.

The initial work at the agency had a lot of language in it about connecting care, for people across the continuum. There was a lot of thinking and language about public and population help --help. For those of you that know the origins of the office, people like John Lumpkin NCHS at the time had a hand in the formation. People from CDC and the public health service. So our background is about population --help, equity and everyone in. Also about connecting on behalf of people, consumers, very consumer focused work. It's called coordinator because of the expectation of coordinating policy and budgets, and action, across the federal government and the word national, it is not federal, the word national means we work with the private sector to see that we are supporting, guiding and enabling the marketplace to continue to advance.

It's a really very important set of complex ideas about our history. To inform our future with the kind of work we are taking on.

In 2009 the high-tech acts gave credence to the office of coordinator as a national staff position within HHS and provided is a set of authority and responsibilities around health IT, for example to develop federal HIG --H IT strategic plans across the government to continuously staff and monitor progress and help at the information technology across the country, to create an opportunity for there to be a government structure for health information exchange across the nation. And to develop a partnership with CMS, a program that would structure, certified and the final electronic health records so when they are bought by providers they know what they are getting and hopefully work in similar ways to improve care of individual patients and ultimately a bigger population of health.

And the new set of programs include the meaningful use program, which is now in stage II. A lot of people are still in stage I, --for about 2 1/2 years now so federal budgets can all run concurrently. Meaningful use is a program that again has its thinking going back a decade, about how we can --electronic healthcare adoption for this nation, for all providers and encourage them to use it in similar ways. CMS leave the program that we inform our technology, enabling support. And so far pushed about $25 billion out of the country so that we can help hospitals, eligible hospitals an eligible doctors adopt the best practices.

The word eligible is critical they are so put that in the parking lot. As we move forward as a country I am conscious of the fact that if we want to move forward as a country to achieve a three-part aim of better came that's better care, lower-cost better help, by virtue of the definition of eligible we have excluded some critical parts to the care continuum from being part of the digitized healthcare structure. Non-small players like long-term postacute care, which is about 40% of health hospitalizations. Behavioral health were patients with significant or severe mental illness are cared for and are significantly more likely to die of a chronic disease or even if they have minor depression, more likely to have more ability --morbidity and mortality.

This is really driven adoption it's only for certain eligible providers.

That program brought us from a place where about 50% were using health records and now 70% are.

It's taken hospitals to about 97% or 87% of Medicare discharges being an electronic health record environments pick a little more than 90% of hospitals have adopted electronic health records. In essentially every prescription in the country, that is not in the Contras system --is electronic. We have been able to use technology in additional ways by creating expectations of the electronic health records, that will reduce errors from here --handwriting, moving across the information highway to support care.

Dramatic advances in a short period of time. We also were given funds to the high-tech act to support grant programs, frontline providers called the regional extension Center that helps practices adopt their practice to change workload and support and encourage the providers and the members of the medical home team or practice to understand the potential of HIG --H IT. And every state in the country, help them to advance their notions of, hardwiring but also issues like governance and how they might use data to improve care and populations in public health.

The spotlight 17 communities in the --program are given as additional funds to see if they can come together across what was typically competitive relationships and focus on improving population health together define what that might look like if it's going to be about improving diabetes, who was the population, how we work together to align our policies or practices, expectations, to get to a place where we are changing the culture around health IT and thinking more broadly than what's happening in the individual practice. In a series of workforce and research programs designed to build capacity and answer any questions around privacy --D talk about a 2000 --a $2 billion portfolio, that work is coming to conclusion are has no cost extension continuing in some way or another. Our era of ONC is starting to come to its end. We are now 10 years old, five years after all run, having to take a breath, stepping out of the meaningful use program, the are a funded programs as a grantmaker and regulators, and reflect about where the country is going in the next 10 years. Building upon what we've already had success wise.

I'm still not sure why I have a job. [laughter] But I will tell you what they tell me, and what I believe is true, if you will allow me. Some quick stories when I was a young faculty member I was asked to run the resident clinic at Kerry hospital. I hated outpatient medicine because it was so broken and painful. I don't need to belabor the point. This was about 1996 to 1999. I was doing a health service Corps charity so I was forced into outpatient services by the National Service Corps program, to do 80% of my time to[indiscernible-static] I were to come on faculty --worked to come on faculty to not only do my National Service Corps that I have volunteered to teach, but I also wanted to do inpatient medicine so bad. And also --[indiscernible-static] As my consolation prize I got good. And also with the medical records committee for the hospital, and the resident clinic they didn't spring that on me until the second year. It was the best thing that could ever happen. When something is so broken in your responsible for it you have to focus on fixing it. What was broken was a long list, that IT is a piece of that. In that environment at that time, the medical records, about 10% of the time the patient would take off work, take a couple of buses in the heat wait for hours to see a doctor they've never seen before, who didn't have any information about them at all, and be told sorry, we don't know what the results of your echo were so you have to come back. If they couldn't get back in three months, they would have to reschedule it was a year later before they could get back to the clinic. If people wanted their meds refilled they went to the emergency room. And the emergency rooms were full of people in end-stage disease which work totally present vegetable --preventable.

I helped because I was chairing medical records, and we owned some software that would in an automated fashion talk to a scheduling system we owned. In order medical records so that person --and another person had to verify it and then it had to be reported upstairs. The human error option wasn't it was more the automated, and 90% of the time the electronic health records were in the clinic and the people needed the service.

And also --we didn't do anything different, we didn't add staff, we just did a Q I process and figured out will cause analysis. We figured out what was wrong, figured out we had some stuff to put to use, and hoped we could get it together with the technology. I think with that we started to think not only a systems level but also that humans could be supported and enabled by technology and it doesn't have to be that complex or expensive.

And several other defining moments, this thing called Hurricane Katrina. When I had at the time, I was still in general medicine so we have responsibilities for teaching. And I have my own patients at the time as well. I had the experience of the whole health system shutting down for months, not having information electronic health records, no registries, knowing our patients were without information about their healthcare needs and regimens. Not knowing how to find them or know which of our patients needed Coumadin and with dose.

And this happened all across the community and is still too much. The nation not being digitized, not mobile, not accessible, and can't really have it there for people when it matters most.

We do a lot of work in Indiana to work on wrapping up a we to adopt a different line so we can digitize the health information, make it web based and move to a place where it can move to a regional information exchange working at the state level to lay the track for health information exchange. And a beacon project also to work on population health.

We did it independently but with a lot of ONC support. --You may not know, I called David Brenner right after the storm and said I understand this thing called --. You do electronic health records and we need --we may as well start from scratch. It will probably end up here because I started with ONC in 2005 and I seen how we can empower on the frontline, changing the lives of the people getting care and the lives of the people providing care. I also have seen how we can really inform the population of public health, payers and the inventive and delivery care.

Bringing all of that in my heart and my mind, this job, helping the team who has been working in some cases for years solving a lot of these issues and I think our timing is great there's a great foundation laid in this company --country, wired across the nation in many years we were not years ago. There is data waiting to be freed and moved across the system or aggregated into data to be put into important uses, for position medicine to public health preparedness.

We have aspirational models, that we have supported or the country has that shows us what it feels like and looks like when it works really well. Played --places like Indianapolis, Kentucky, Minnesota, wherein it does move and in is enabling supportive care and health, and he can make a big difference in people's lives. We also have not just aspirational models but technology that can be more widespread, adopted more widely.

Standard tools like fires, codes that have been around for a couple of years, and allows us to create better interfaces for electronic health records, to grab data from deep in the system to use for important purposes. That did not exist, and has been invented and in the maturation process.

We have payers and providers who have now some experience with some expectation, that we can use the opportunity of digitizing and sharing information to reduce waste and redundancy. Therefore lower cost and have opportunity to not just value-based payment but they care more convenient and accessible for the folks they want to serve.

Increasingly related to employers. And a set of vendors who compared to when we started with high-tech procedures a few years ago, our were ready willing and able to be a part of the solution in terms of sharing information and being part of that future of the data and health information exchange.

If you haven't noticed I'm in the extremely optimistic person. If you had asked somebody else they might have a different opinion, but I sent around this country and I've been doing a lot of listening, and that is what I hear currently from all corners. Everyone is a little further in different places on the continuum that there's a bait --great deal of readiness and I had a tremendous experience last week with subject matter experts around the country, the interoperability, this is how it happens when you have a dropping dreaming about it.

Just to see what they would say together, right they tell you individually it's one thing but when you put them around the table to pose questions to see if they say yes we can and they said yes we can. They nodded their heads and walked away and I've gotten lots of terrific feedback on how they feel the timing is right, where we can go together and solve this daunting issue of interoperability.

So what we focused on now, resetting priorities in general. The federal government, I think my big take away for you all in that space is first of all, the resetting of priorities is around redefining the federal H I T strategic plan, reshaping that so that our goals, were trying to get to isn't just about improved care, it's about improved health.

As you all know, better health involves more than the health care system. Our vision for the future is a platform of health IT that more inclusive of other trading partners, housing, education etc. housing, education etc.

We will be stepping up our thinking about the other policy and programmatic and regulatory levers the government has. The waning of the AURA era is a gift, it helps us to --as a payer a sign that says evaluators, and consumers, to understand the long-term sustainable levers we have for this adoption of interoperability introducing data into the future.

It's also a chance for us to go deep really quickly and a place where everyone has agreed to which is interoperability. There are many issues at the table for health IT customer frontier issues, some are biting at our heels like adoption and meaningful use program. Getting the data is the way we cannot just push but pull. We start to increase the demand at a real-world use case of data, that happen increasingly and have the potential to awaken even further. Consumers, innovators, payers, the delivery system for example, to want the information to move.

We laid out a vision around interoperability in June. A call to action, a call to the table to invite all of the stakeholders to talk about interoperability and how we can get over the barrier together.

We have been going deeper inside of ONC to align ourselves, you would be quite amazed at how much work that is. I'm sure you can imagine, the definitions around terminology, like interoperability, we spent an hour and a half just trying to define interoperability. There's an engineering definition we use about exchange and use of data, the use part being a critical word.

We are now starting to expand our circle, we have subject matter experts from the private sector working simultaneously with the federal partners. The ADO deep, FFA, CMS, Indian health service, FPC, what is interoperability mean where are the barriers to get over that. And what are our goals, to get where we set the three-year, six year and 10 year visionary goals. The 10 year being learning the health system, where there is opportunity for continuous feedback at the micro and macro level to improve care, science, quality, and make it a much more seamless experience for everybody.

I think the challenge at the door is just getting the healthcare information into one place and moving to another, quite frankly, and it's doable but we are setting our sights on that within 30 years. And the building breaks --building blocks, the out --we sponsor abilities we have, who governs the road, how we take the governance structure that exist now and we've been together into something that make sense. The private and security challenges we face, not only for regular data that a resources from outside that are not hit the covered, when we want to enhance privacy. And how we we pay for this. Had a we create a sustainable infrastructure that recognizes this is a public good, that is necessary not just for healthcare but for health. An opportunity to build a research case, --case implemented care case, on this highway we will build together. And make it something that is flexible enough, and is infrastructure and all the way described, that it will meet the future that we cannot imagine.

That's where I will end. Pretty good, 15 min. I will give you some imagined things really fast about the kind of stuff that comes at me in this job which is really superfine.

--A whole research portfolio on wearable underwear.

Just tell the students don't Google underwear technology at your office.

[laughter]

It's one of those things where you're like what is she talking about? This is an emotion --where technology is right now, the kind of thing that captures the consumer or caregivers of imagination, where let's say you have a nursing home and you want to be able to have wearable underwear to monitor urine output, glucose, there's respiratory rates --a trial with folks with COPD, sending them home early from the hospital with wearable underwear, that can be some remote tracking and it's more comfortable than having --.

This is completely regular technology. Things that can be done today, wearing one as we speak. And as I said it's not that the coal country but the country likes it. The industry in the space is growing fast. There are challenges in our world --where did that blood pressure management come from is now in the medical record pics the the the frontier issues we have to build a in the data. And for the privacy and security around it. How it will flow in and out of the system.

And the interesting thing is the wearables. How it changes how we interact with patients and people, as they try to live their lives.

An interesting space were not really ready for from a policy and technology framework, is interoperability moves from the notion that EHR's are talking to each other, sharing data between health systems and health care systems. How do we create on and off ramps that allow for something like that help databank were patients ask perhaps a third party, health system, to host their data for them. The patient centered model where I could choose to send information from a cardiologist, from my fit but, from my remote-sensing device monitoring my glucose continuously, and the tips that are in the pills that I swallow to give a full picture of my overall health. I may not want all of that to reside at Kaiser because I may shop or have other sources to look in.

This is an old idea. I don't know if you know Bill, he's written a lot of stuff in this area. Nobody would ever do that, like his crazy, but he's still around in pretty happy right now. There are some big corporate players thinking about the space. It's no longer RND, the have information the way they have their financial information. The EHR stuff doesn't go away, but it's another way to handle data.

And I will input this comment. We do a lot with AHRQ, and they bring to us the scientific knowledge that we can apply to the policymaking that we do. They help us to cast ideas on the front lines, and they help us to move into frontier areas and as you all you may know the last year the project with adjacent scientist, who for the first time did and HHS progress this process, putting smart people together to solve interoperability, he didn't know anything about electronic health records and I'm thankful for that, because at the end we have something that looks more like a picture of the future, completely not predicated on the healthcare world but on the whole world, something we have embraced and and are acting on. In our purses of the --partnership with them. All of the work that really matters. My pitch to you is please let them keep doing that. It is important work we don't have a scientific foundation and we want policymaking to be part of today but also into the future. I will stop there and take questions.

Inc. you for coming out and chatting with us. A few questions, a number of questions. I will start off with one. You mentioned central Indiana the health information exchange there. --It's about 40 years on now. I live in the Washington DC area, where nothing talk to anything else in terms of health information, across three jurisdictions. I recently changed physicians and nothing moves with me.

I do we get this community or a community where Indianapolis is or central Indiana, without taking 40 years?

At the risk of sounding flippant, I wish I had the answer. What we do know, I'll tell you how we are trying to reshape some of the thinking. To even more dramatic I was in Alabama and a listening session with 40 stakeholders. --Montgomery Alabama doesn't have broadband access, so adoption is their big issue. There's not much to share yet because they haven't gotten to a place where they feel comfortable using EHR in practice. They are moving but they are not were other places in the country are. On the other hand Minnesota, about 10 years of organized conversation but --history. A learning health system, how we incorporate social, how we use the data for customized care for patients. It's a completely different part of the septic I'm --Ekstrom.

Different part of the spectrum.

The countries not in the same place with payment reform, care delivery models the information available and able to be shared, and with the consumer and provider education engagement. Our policymaking has been to really push, push, push. And what we're thinking about is why we keep pushing, how do we help raise the floor and not leave to spine.

--Folks behind.

Leaving people behind is not in my DNA. We have to figure out a way that we can support folks who are not quite in the space. That's why interoperability is the tie. The conversation that make sense to people who want to have their information portable and move with them. Or for there to be information to move you have to have a doctor. If your doctor uses that information to take care of you, yes but the incentive properly, but you also have to have a system that you not just the data in but it has to come back to you. The return on investment as really materialize for people in the front line. There is an intense pressure that we are all feeling. Congress is ensuring that we feel, that we really show some return on investment for folks. That's why the interoperability piece that I mentioned is a way to we can to bring all of these issues together. It is so solvable, we can do it and it's not as mechanical as the adoption, at the same time.

Specifically, we are working on thinking through a continuum, and assessment of communities that is a continuum. So I could say a place like Washington, you are you are on the scale and the culture and policy and regulatory change that will make a difference in your community are these five things because we've seen other communities come from there. This is what we recommend. A state-level for example, that's how Minnesota and Indiana have done so far, they have not waited for us they use their state. From a Medicaid participation, certificate of need cop public health fund to advance H I T. We are cataloging that, convening states in August to begin a conversation with the government, so what they can do like --locally based on their own continuum. And also at the federal level how my blunt instrument doesn't make people blind.

A number of questions. Andrea, then Henry --

Thank you for coming today. Thank you for coming to AHRQ, we hear great things about you . That said, I notice this is a loaded question. You talked about the public good. How we might pay for this. How do you think we pay for this? That to me is always the holy Grail. I would like to hear your thoughts.

I told you it was loaded.

That is a nice question. I did tell somebody that public comment and in Washington it is so lovely. Nice letters. Where I have had public comment in New Orleans, with the band showing up in city Council. It's all relative.

That's a great question. Right now I have an opinion, not evidence-based informed opinion. So looking at our lessons learned, the ONC lessons learned from the health information exchange grant, $560 million in grants across the country, looking at how those --have were to sustain themselves and how some of them are more mature. HIE.

We've been unwilling the use case with the highway, the services with the highway. The expensive part of the health information exchange is when it becomes the health information organization with the analytic providing that back to providers as an example. The expensive part is the highway, the massive highway. Or shiny that they are doing in New York or what they've done in Utah. They think of it as a public utility, not in a regulatory sense so don't misunderstand that were public utility, but that is the public good in there's a price point for everybody in the community funding the stretcher. And something --structure. Something to wrap around the government and privacy security data. A I don't know the number. I will tell you a number for fun. It definitely is less than two dollars per person per year in the country. It's more like a buck and a half.

I really think the the highway, just the highway, think about your highway system. Again, not the other stuff on top. I believe philosophically we've been working to the healthcare system to pay for that and I think that's not right. It's not right because we've been using a service model to pay for it if we do transaction-based with the --interoperability, it's a perverse incentive, you don't want to use it because you get charged. You wanted to be free for primary care, just use it all the time. I actually said that. [laughter]

I couldn't resist. And then you want to make it, think of it like a public good, the language around opportunity for us to be inclusive of the social determinate partners, public health, it's the liver it because I think that changes your focus. It's not healthcare's responsibility, it society's responsibility. Once we get down to the basics, and P along all of the --things we want to do, we will realize it's less expensive than we thought.

Henry.

I do want to thank you very much for being here. My question is in line with the goal of AHRQ to address have --help disparities and underserved populations. I had the great opportunity yesterday to be at a White House event talking about the affordable care act impact on lesbian bisexual individuals, and how to enroll. Here we have $8 million we enrolled and a significant number of those people are of a sexual oriented identity. --Backup me thinking that we still have this challenge in the functionality of meaningful use part 3, were having discussions about inclusion of some measure or metric of sexual orientation or gender identity hardwired into the functionality of the electronic health records. But were not quite at that point where there is that all to action that needs to happen. We have more and more people who are coming into the health care system.

You can look at this from two parts, a research perspective, and we don't have data. And HS says sexual orientation data was just published last month. We have nothing on gender identity or gender expression. So resources would be important. But I think more importantly we have issues related to patient safety.

There was a great article written last year in the JM I a, by Maddie Deutsch. I wrote a response and was surprised when it got published later this year. One of the things I was concerned about really has to do with the --for trans people. We have binary types of health records, mechanisms. It does not allow the practitioner to effectively care for those patients, nor bill or charge or any of these other important functions. I hope that the ongoing work will also continue to move the needle a bit more, in terms of progress in this particular area, for many different players. --Organizations that are very interested in the work we will continue to see more people coming in. These issues and things will become more common, not only for those of us like myself who provided care for number of LG BT people but more commonly throughout the United States, wherever people finally have health access and begin to use it.

Thank you. The issue of sexual orientation, gender identity captured in the electronic health records has come to our attention. And structured and unstructured ways. The team thinking about meaningful use part three taking that information, and had some conversation with --Harvey, and others who helped us. --Will resolve it all and in the next iteration, I am not sure, but at the national Library of medicine they want to help us all that in a way that may not be all of the expectations, but it gets is further because there is some language opportunities, some ways we can do that.

On the other hand, an issue I want to make sure this group knows about, which is we are not always binary. It's not just about gender. We have quite a lot of option Alan the end dated --optimality in data collection. We don't have a data dictionary. We might have EHR is entered as an letter M or a zero in a one, in", the interoperability piece get this to that atomic level and that's why we need what we talked about, things we can move with that is funded until important work, not just for interoperability, but for quality --for example the safety issues you describe. You do need to know exactly who the client is in front of you.

We are supportive and we are trying to solve it from a technological standpoint, not just cultural pick

Harry.

We are glad you took the job. I love thinking about the 10 year thing. But a question about closer research, and clinical care both for healthcare, --social equity, this is a complete change. I look forward to that 10 year vision. I actually think this infrastructure for research, is transformative. We we do research is fundamentally more powerful for inclusion, going for effectiveness in the whole populations. You know the stream so I won't go on, but working with --cornet doing so. There's nuances between them and actually as you probably know is the TSA's are doing a good I threshold thing. For the other 4800 hospitals in this country, which is part of what we need to accomplish

3600.

However many, don't have --this year, I to be to. Most people get their care. Our only going to get them into this research data warehouse environment? I know you can say in the future it all on a be one big warehouse, but this year we want to have, maybe for the next five years, data warehouses that help us identify --all the stuff you've heard about. I am struggling with smaller hospitals that want to get involved and they can't afford to get on board. Are you following that? What is the solution?

It is not a tomorrow solution. The good news is that through P-COR , HHS is working to stay alive --aligned with P-COR , but on the other side the common data element work that we are doing, which gets back to his point, the core common data elements how we define them, are they standardize enough that it's not extra we --extra work to do a data feed --M.D. pulled more easily with permission from the patient. I think that the worst thing we could do right now is that any burden on the frontline clinicians, nurses or doctors, in terms of expectations of documentation. Whether they are being paid extra or not.

We can argue about that. I think we have to find a way through natural language that processing through whatever we can find, to pull that information in in a way that is already happening for some folks.

Technologically it's feasible but it's not perfect now. We want to work in that direction I cannot see it being a big cohort of the US overnight. That we will have all of those things in the measure. It's back to the data thing, in the infectious disease folks, anyone here? This is how I think about this problem you are describing. It could be that simple but when you get into standards, the dimensionality of the variables, all of the ways you can describe in allergy, a good doctor has 20 dimensionality is --knows what the reaction was etc. You can get into the standard world of creating all that, and go down this rabbit hole and not done the other standards.

On the other hand, --start with that. And the research may not be enough. But if you start with but you can learn --[indiscernible-static] What I'm trying to be clear about, getting to where we have the standardization Mike the name of the drug and the way it captured. I don't want to disappoint you because those other technological ways to get over it. My cautionary tale, I know you know --[laughter] AHRQ people . At least --you need a pen or a T-shirt with that.

Just my worried that will have all this data and the turning and running, like blood pressure management, having an appropriate age and another thing to let the bank know they have diabetes and not give them alone. --A loan.

That can happen today. We just have to be really thoughtful about what were asking with the data, we will use it for, and no you are inclusive of the patient and consumer in the patient making conversation.

Can you have a 100 bed hospital in a small town, if they want to get involved with this with research now? That's what I'm dealing with, I'm feeling, finding an interface maybe.

They can pay for an interface that would make the systems talk to each other. It will not be seamless get.

--Yet.

Just to echo everyone's comment about thanking you for it being here. --Just for context, I am Padre --Patrick Conway's technical advisor and we work a lot with your staff on the QIOs. Thank you for that, but a lot of good work in demonstrates we are a quality in the --roles coming together.

A couple of comments. You mentioned or discussed and or operability, not so much about portability. I wanted to hear your thoughts on that. Especially with the imagined stuff, excited about the health databanks. That actually sounds cool and scary as it is, I can 1984.

The other thing is thinking more about the health disparities and how HRT can help address that, and what we at CMS can do more of to help move that agenda forward.

The data portability issue is a barrier that things like usability because you buy a product and your in a covenant marriage and it's hard to get out because you can't pull your data out. That's how --fire might be helpful. And how we work with contacting to support providers to tell them what language to put in a contract that allows to pull data back out. There is increasing pressure for us to think about other potential lovers at we might have, certification or otherwise. It's a big issue. And it has to do not just with switching provider systems that creating competition, patients being able to pull their data from somewhere else work that's where the databanks folks can get jazzed up about that. It's not just one guy.

And I'm speaking to them in Washington if you want to come pick and I don't know when, in a week or two. And the second part of your question, I think IT is a level --first of all it's a multidisciplinary skill by its nature. I love that about it because it's a great place to come with health professionals and others together.

But it also can be a barrier and it can leave people behind if were not careful. We have a lot of time as I mentioned focus on rural and safety net providers. I've asked about a meeting to learn more about it. The SQ AC office has done we --great. The tribal providers have been great. If you look at the data now, it is small provider practices, and it is a cohort of about 60. There's a lot of minority clinicians the don't to like this is for them. They tend to take care of minority populations. That is not okay. Nadine Grassi and I were talking yesterday. I wouldn't say colluding, but planning to open our tent a lot more. I'm sorry we haven't talked about it but our tent is not --diverse if at ONC, and it has to be. There needs to be a broader set of voices at the workgroup level, the policy table, in the office and on the front lines. That is clear to me. It is something that when I walk out the door I want to have --

Jed.

Thank you. I'm curious about your view of the future of the regional extension centers. They have been remarkably helpful and successful. Will they continue in their current role or do you see a broader use for that infrastructure?

I think it's one of the most valuable assets we have that HHS. It is a direct connection and a database of about 150,000 providers across the country that is an opportunity a database. A tool that with the snow in the continuum of payment reform, in the medical home and the patients they serve. It's more than that, if the feedback. Away that frontline providers, when I go to a place like Alabama, or they can pick up the phone any time and tell us, this is not working. Were trying to do transition to care in meaningful use 2 and it's not working. Were trying to make it work in this is what were doing, is that okay.

Without that we are blind and were making policy without a tangible feeling it there like a non-no moral --nonnormal cohort.

And to keep them, the next iteration has to be more inclusive of providers that are having and adoption. That means a lot of things on the front line. It has data mechanism where were not a granting authority but it's a partnership. They have to feel free to give us feedback about our policies and programs, what's working and not, not just happy stories but the bad stories.

I think also of it as a chance to build local capacity that can become more intended longitudinal and workforce development, because I think care in the delivery happened of a local level.

The data I would say is 85% sustainable over time. Sustainable means because they have a business model, that causes them to collect these are defined in --funded in another mechanism, for folks that likely need their hope --help the most pick

To give a few solutions works, but nothing is solved yet.

Thank you Karen. The last question, the next session after you is going to talk about what we have been doing in HRT --HIPAA --H I T, in a variety of areas, producing research and evidence to try to figure out how this can be better. We have not done a lot of work or any work around interoperability. I wonder if you have thoughts about what evidence, or research would be helpful, in moving forward the interoperability agenda you laid out here.

. So there's a bucket of work about the evidence-based to the policymaking in health IT, it moves very quickly and research moves more slowly. It is a challenge and we often do the best we can using qualitative or Delphi methods to figure out what will work.

There's another work happening already in the country like in Indiana. We can learn about what use bases are relevant in terms of improving care or and improving health of the functionalities people are using. There are some, a lot of work that needs to be done in a space about human machine interaction. What will drive consumers, buyers to use the system, do not just have the capability in the place technology --technologically but to use the systems. What are the drivers, is in all payment, workflow, usability of the system, a single sign-on? A whole bunch of questions from an analytic standpoint, but will have the most impact if I have a short amount of time and a little bit of money. If I wanted to do one or two things, what would it be to make the most difference in terms of willingness to use it. If we build it and they don't use it, New York is a great example, 2.4% of physicians when into the information exchange the pullout information about patients, even though it was widely available and achievable, that is nothing.

We want that to be 90% to realize the benefit.

And then a set of challenges and questions that have a longer-term horizon, some of the stuff Harry was raising and others, about what are the thorny issues in ways that folks could solve --multifactorial problems having to do with data segmentation and privacy, sharing behavioral health, others like sexual orientation --. Some issues about the interoperability in the delivery system, but beyond. The learning health system, where do we start.

And the safety peace. This is really top in my mind. If it wasn't, the Boston Globe made it on the top of our minds. And I think, they raised a story this weekend, a story we have been following and working on with you will for a long time. What are the implications would you introduce a new technology into a complex emolument like healthcare. --At the end of the day we have to really understand that we want to create an opportunity for there to be discoverable data based, were like the aviation world has been, as we move to that space to understand what we are learning from that, applying it back to the field and see what changes make a difference and the use and application of IT. That get into some --expression. Genotypic expression, you can really help us understand what happens in the field. How it affects care and practice and research etc. care and practice and research etc.

Thank you very much.

Thank you, this was great. [applause]

--tran07 types.

[laughter]

-- AHRQtites

While there setting up, --Teresa the chief of IT research and AHRQ . Teresa is probably the only PhD we have in industrial with engineering and the agency. It brings a unique perspective to help IT research. And she's going to talk with us about the work we have been doing at the agency. In health IT. Over the last decade actually.

Is John White still on the phone?

Hello from Tampa.

Hello John. We are joined on the phone by John White who runs the IT portfolio, on a very well-deserved vacation. And not as smart as he's cracked up to be, but thank you for joining us John, we appreciate you being here.

And thank you Rick, I am Teresa and I am here to help. The put it in perspective, Dr. Sala spoke about ONC in their mission. I wanted to talk a little more about the portfolio, but we have been doing and where we're going. Our mission is to focus on the evidence base building how help IT can improve polity and safety of healthcare. We also service the science partner which alluded earlier to the sister agency. For instance last year we awarded dozens of grants and projects to evaluate the stage III meaningful use objective. Were working on that with CMS and ONC and hope to have results this fall.

We are also 10 years old. A little bit older than ONC. And we have funded projects across 46 states, Washington DC the budget has fluctuated. This year we have appropriated about $25 million with a $4 million increase for safety projects that I will talk briefly about.

And I just wanted to give some examples of the kinds of works we have been leading and funding at the agency. We see our role in three types of approaches to build and disseminate evidence-based on health IT and how it improves quality.

The first one being, funding demonstration and feasibility projects to find new solutions, to see how they work and how they impact quality if they work, and what about them makes a difference in terms of quality.

The first one I want to talk about is actually a set of groundbreaking projects and initiatives funded several years ago. A five-year contract to demonstrate how to set up and conduct health information exchange. And the first set of large-scale demonstrations, as stated regional data sharing with the interoperability activities, and it yielded lessons learns for other HIE , funded by ONC.

We also funded groundbreaking work in clinical decision support. These demonstration project ended in 2012, with report and videos that came out yesterday, from lessons learned from those projects. They really were trying to demonstrate how to create tools to translate clinical knowledge, and guidelines that can be used by multiple help record systems for implementing decisions across a range of healthcare settings, bringing that to scale. In addition some technology created by these demonstration is really been used by the national coordinator office for help IT as a basis for clinical decision support standards.

This is a smaller scale project. A project we are excited about nonetheless. Grantees in Oklahoma, with Dr. James --develop a personal health record for wellness. After the study ended they were able to, they showed 84% of adult the 95% of children received all recommended preventive screening. Nobody in the US is getting those numbers.

That is a great example for the field. And the personal health record is now available throughout Oklahoma to all primary care practices and wellness organizations.

To to speaking of safety. The portfolio funded development and testing of the health IT hazard manager, which is a software tool for users with help IT for safety. This report addresses concerns that were noted in the IOM report on patient safety and health IT. Last November ONC actually issued a guide to improve reporting unsafe conditions and noted this is one of the reporting resources they recommended folks to use. The other one was also a AHRQ resource.

In addition, every patient safety organization is also using this hazard manager to monitor safety for clients.

This particular project, E, which stands extension for community health care outcome, was led by --of New Mexico. This has provided access to patients that otherwise would not have that access to that expertise pick it was found to be comparable to usual care, and it spread rapidly which started with treating hepatitis C and spread to other editions, mental illness, chronic pain, pregnancy, and a dozen partner sites. With the Johnson foundation and the VA among others ECHO. That was.

Dr. Carol at Brandeis University not only was able to demonstrate safe and secure electronic --controlled substances is feasible. But this project was also able to inform the DEA development of the in term rule published in 2010 allowing controlled substances to be tracked electronically for the first time. When this was awarded we had to get a special waiver for the grantees to proceed with the project because at this point that was not allowed.

Moving on to the second bucket of work, that we find, it includes evaluating and assisting help IT work for determining impact. One example I wanted to show you is on computerized provider and supporter entry systems, which you already know, can process prescriptions electronically and send them to the relevant individual or department. This study used a mix of literature review and meta-analysis methods and found that CPI we may help inform medication errors across the United States. Building the case for using health IT and having a direct impact.

In addition, some of the earlier work, we funded a study looking at a prescribing --E prescribing. When using the electronic prescribing with formulary data, everybody uses it for subscribing, the stakes would add up to about $3.9 million, or for million dollars --

The last group or set of work we do focuses on the dissemination of the evidence, making it available to stakeholders. A lot of that we do through the website and there's a link for that at the end of the presentation. We have other grants and contract mechanisms we use to disseminate the evidence. One of them is a series of help IT opportunities, and we been doing that for some time now. Free to the public, they highlight research findings of some of the products, and the topics are very diverse ranging from an example a few days ago, change management. We will have another one next week on novel help IT intervention. Patient safety patient centered care, quality measurement, and vulnerable this of damaged populations.

--Disadvantaged populations.

We are pleased that we track from these teleconferences for academic healthcare spector as well as vendors.

The other activity that we do to synthesize evidence is we look at what is being done and what we know about how effective help IT is. We find evidence such as this one to show where help IT is currently having an impact and where it remains to drive future initiatives. This evidence was published recently, focusing on clinical decision support, and knowledge management system. While we have strong evidence that CBS can help measures, we thought we needed more work for the effect on clinical outcomes as well as economic outcomes.

We also have some resources made available to different stakeholders to help with either help IT or help IT projects. One of them --health IT projects. This is an available registry of healthcare related metadata. Late last summer the meaningful use clinical quality mellowed --measures that eligible professionals in hospitals can select if they meet the requirements, --an effort coordinated between us, CMS, the national Library of medicine and ONC. You can see across the different portals for different types of initiatives that are supported.

One of the tabs correspondence to the children's electronic healthcare format that another good example of collaboration, this one between us and the centers for Medicare and Medicaid services, the format was developed as part of the 2009 --legislation. And includes child specific requirements in electronic health records to perform optimally for specific needs of children. As you may know the EHR's have not been designed with pediatrics in mind. This is an effort to nudge systems in that direction.

It was released publicly last year.

And very quickly, Dr. Szabo talked about this project which was I wanted to highlight in the extent that it makes sense as she alluded to earlier, pursuing relevant research and interoperability and collaboration with ONC

Moving forward, we are continuing to support research and help IT --health IT and how it improves quality. We currently have a funding opportunity announcement for fault --small feasibility studies. We also support larger demonstration projects, were initiated research all the while trying to bring about the next generation of help IT --health IT research.

In addition as I mentioned earlier, we had special appropriations this year for project focused on the safety of health IT systems. A published in February, and we hope in March to have --awards in the fall.

I believe Dr. --mention this earlier, the Center for --outcome initiatives.

Thank you Teresa. Before opening up, John is there anything you would like to add? He has been directing this overly over many years.

Thank you Rick. The only thing I would add is that you had asked during the presentation --interoperability. We actually have a large systematic review underway, looking for evidence in the literature of health information exchange on outcomes. I thought I would add.

Thank you John.

And.

A quick question, I'm wondering how --relates to your work here at AHRQ . The patient safety organization, given the issues with interoperability, and the background.

That's a great back --question. --Another part of the agency. And I don't have --

I'm not quite sure about the question quick

Given the importance of the conversation we just heard from ONC, and the need to collect this kind of data around safety and quality. And what I perceive is really sunk at between the different --gaps between the functions and initiatives. To feel that what were meeting is more integration around the patient safety.

I think you probably know we have been working diligently to develop common formats for reporting adverse events. Encouraging, we are unable to require but encouraging BSO's to use those common formats and many of them are. We've also been working with ONC to have common formats as being part of EHR's, and that is, John I don't know if you want to jump in moving forward. It's still a work in progress.

The only other thing I would add, is recently there has been a lot of dialogue at the health IQ policy committee, and they reported out to the full committee about the best ways to gather information --. They came to the collation --conclusion that patient safety is an important source but not the only source for a full-blown discussion on the as oh --BSO --that movement take shape, I think PSO will definitely play a role.

Jed.

I have two areas. Medication reconciliation. Meaningful use tries to capture in a very unsophisticated manner. The screens for research on how current medication uses care elected --collected. Understanding the process in a variety of things. That's a huge area. And the second one is automating the extraction of beta quality elements for participation in improved initiatives, or even joint commission core measures. At Kaiser Permanente, we worked up automate those things and had success from 0% to 100% depending on the nature. Using highly skilled and paid nurses to extract, around selecting data is a barrier and a waste. There needs to be an automated way to do that.

Thank you, that is a great suggestion. Under the inventory 16 quality program, and the management globally, if I'm hearing correctly you're asking us to take us that back and understand the process before it's automated. We have been pursuing work in enabling quality measurements, and that's something we will take to heart, thank you.

Tesoro, Harry, Tesoro, Harry,

My question is around the health IT finances. The focus of the conflict is to get information about polity in IT or quick

We essentially to disseminate lessons learned from the projects we fund, we pull out teams based on, what the projects are doing, the needs we see, in feedback from the participants in terms of what they would want to hear about.

And the QIOs or kilos, are they part of the imitation? Do you have an extension quick

Anybody who signs up for the list serves, we also publicize it on the website.

This is John. Briefly for what it's worth, we have a listserv that has 28,000 --individuals on it. [indiscernible]

Thank you.

Mary.

--Larry

We switch because I have to head out. I wanted to pick up on the comment about the interoperability with the pharmacy. It is really not very good and not very consistent, starting with electronic prescribing, trying to get a handle on adherence in the number of patients who never make it to the pharmacy to get their prescription filled. We don't have a good way of measuring that now. And even getting into the issue of vaccines and some of the vaccine coverage, not in the pharmacy but it --budget --doing adjudication for pharmacist or vaccine administration, there are all sorts of issues with the pharmacy data that we just lack a good system for.

That is a great point. We actually just by way of example, had a grant that was trying to develop personal health records to support patients and medication management. Originally they were going to try to have a patient interface and a provider interface. And the providers wouldn't participate because they couldn't handle another interface. They integrated a pharmacist and it didn't end up going very well in terms of supporting patients in terms of medication.

A special announcement that Jamie needs to announce.

Whoever owns the dark blue Toyota Corolla, with Virginia plates, I don't know if you are in the room that please go outside immediately. [laughter]

Thank you.


We will go to Harry next. Been --

[Captioners transitioning]

--We have map equipoise, by predicted outcome. What we found recently is when we share this with physicians

[Captioner must disconnect line to transition to new captioner.] EMS records are so primitive there is no place to start -- no intention court.

I think that a check to be enormously helpful. EMS providers are fairly receptive to innovation and the like cool things.

Thank you.

We did have a [indiscernible] where we wanted people to explore decision-making. It is a tough nut to crack. There is definitely a lot of work to do there.

Before going to Sherry -- [indiscernible] is linked EMS records in multiple providers and community clinics in San Diego. It is quite a system they developed -- I hope it doesn't end in San Diego.

Shari?

Maybe I am naïve here and this is not the right forum, but putting the diagnosis on a prescription and enacting that within the EHR throughout the whole system -- the backend data collected the right information on outcomes I don't know who to talk to about pushing them forward -- I know it was an initiative of large employers -- I am curious if that is something that could be added to the list.

Great suggestion. I can follow-up --

It is a disclosure issue.

I me to this quickly -- the reason for my question earlier -- what I am hearing -- what it's like in operations to be able to put together the story of when bad things happen to patients and they -- the sources of the collected different.

Medication safety is a great one. When most of the medicines now are barcoded, when there is a recall you get very little warning. It out. We don't have a good way in EHR to know whether or not -- where the medication when and to whom. Starting to connect the technology that we have deployed to -- is not informing us the way it should. Related to that -- the issue of disclosure. When something does happen, is that the systems now are moving toward full disclosure. That doesn't show up anywhere, at least in our electronic records and we have every platform there is. It's time to start asking the question how can the record become a tool? That would be a wonderful project, potentially, to find, from our perspective.

Thank you very much. I think we are done with questions. We appreciate you coming and chatting with us today.

Now is the time for public comment. I don't believe anyone has signed up for public comment. Does anyone in the room want to comment? I am opening it up at this point.

I gather there is none. Now I will go into the last part -- our wrapup. At this point we want to go back to the earlier question around evaluation. [indiscernible] started to post this. I don't know if we can go back into the slide deck and bring up that slide. We can bring this issue up. I thought it was important enough to come back at the end here.

Thank you, Bruce. Let me start by thanking you for your suggestions on the last session which a variable. While Jamie is doing this, thanks Jamie once again for her work in getting it all here today and giving -- keeping me as organized as possible and helping to make this a successful meeting.

The request I have for you is to help me think about what the evaluation effort should look like. As I said this morning, we are a research agency and the job is to produce evidence. We've spent much more energy on research and evidence about the health care system then we have in producing research and evidence about the effects of the agency activities. Certainly, we have spent a lot of energy on evaluation of various individual activities. There are also some activities that we engage in that we have not evaluated. More activities than we have resources to evaluate. We haven't really done a full portfolio evaluation. My interest in evaluation comes from two perspectives. The main one is the management perspective. Here I am -- a director of the agency. We go through a budgeting process every year and a real -- reallocation process every year. We need to decide where we should be concentrating and where we should be putting resources. In the absence of having as good information I was like about the effects of what we were doing already, that is a difficult set of decisions to make.

Then, I think Victor earlier in the day talked about -- in the past these meetings have often been around how to tell the story of AHRQ more effectively and the evaluation or the desire for evaluation activities in part around that. So, kind of two purposes here. I would say the main interest is on the management side, but there is also the question about externally acing storytelling.

-- Externally facing storytelling.

Earlier I had a slide with some overview of our activities, at least a list of acronyms. It is difficult, I'm sure, for many of you to respond to what we should be doing within evaluation without having a solid understanding of what we do. Many of you have been on the advisory council for a few years and probably have a somewhat better idea. If it is useful, I am happy to walk you through this. But I don't want to take your time unless it is what you would like.

We have activities directed at producing evidence to improve quality. Evidence of improved safety. As [indiscernible] pointed out in an earlier conversation, parts of quality. We do a lot of work on producing evidence to improve accessibility and affordability. We have 2 big data activities or productions of data so that the [indiscernible] project and MEPS work -- big activities in the agency producing data used both internally and externally for research. Investor -- investigator initiated grants -- this depends on what comes in -- this contributes to all of these areas. This provides support to the task force. As I said earlier, we operate the national guideline clearinghouse and quality measures clearinghouse. We've been doing work on understanding how to reduce errors in hospitals and how to reduce the friction in the medical liability process and we brought up the work that we do with patient safety organizations. We talked earlier about the Cusp work and trying to make care safer. I will throw it open here and say -- if you have any advice for me about what and how we should have an effective evaluation enterprise, I would be interested.

Your mission earlier -- I wonder if you would restate that. In a sense the evaluation should serve that.

Sure. Our mission is to produce evidence to make healthcare higher quality. Safer, more accessible, equitable, and affordable. And to work within HHS and with other partners to make sure that the evidence is understood and used.

So, we produce evidence and try to make sure it is understood and used.

One thing you want to do is find those factors. You will not change healthcare with your agency here directly. Indirectly there are a lot of vectors. To the extent that you touch those actors in HHS and I'll swear, that is your outcome.

I wonder if you could work back from that.

I didn't hear the word innovation in that mission. I hope it is implicit, of course.

Yes.

It seems like backing out of that would set you up with a set of metrics for the impact you really want to have.

Yes, for each of these, do they produce [indiscernible] -- it is understood X

That comes from [indiscernible]. That could be initiated from the trainees -- they provide the pipeline. Ultimately, it has to serve that.

I will add something which is probably obvious to you. I assume that all of these things -- I suspect more than this -- they have constituencies in various places. To a greater or lesser extent, some of them. My only advice in addition to Harry's would be to spend your limited evaluation resources on the things that you can influence and maybe not the things which for better or worse are going to be harder to influence. Regardless of the effect -- the relative effect.

Some political savvy as you go forward.

Enough said on that.

Any other comments or thoughts?

I guess I would also add that the timeline -- a good sense of this. How long this initiative has been going on.

[participant comment - no microphone]

Then you have to make a decision about whether we should continue or discontinue [indiscernible]. Maybe this is an opportunity to go back. [indiscernible].

You put it up as selected AHRQ activities. There are things that are not here. I wonder how you got to that.

For the most part, these are areas where we spend significant resources. There are a limited number of things we could squeeze into a box. So, there aren't many other things that we do that are not here.

It is quite odd in size. As you heard, the HIG portfolio is a $29 million portfolio. [indiscernible] 14 -- we thought at one point of trying to scale the want to size. It seemed like beyond what makes sense for this.

Another. Thinking about this -- I love the idea that you are trying to inform. Provide evidence. Some of this is acted upon so we can't just take evidence of impact only. For example, your annual report on disparity. We wish that the nation would stand up and say holy cow, we need to do something. It is still very important that you provide that. So that they know what you do.

Some of these things will have -- providing that evidence is a unique role but you should do and see how well you're doing in -- have it judged by that. Others will be what they are impacted on.

You have a great portfolio. You need to be able to test value.

This is David Ballard. I have a comment. Rick, this is not something that you have already thought about. I think having greater clarity about the role of [indiscernible] relative to the role of AHRQ. The clarity there -- that would be more effectively and efficiently communicating the clarity -- I think that would be helpful.

Thank you, David. Joe [last name indiscernible] is one of my favorite and most frequent contact. At the staff level as well. We are making good progress there. Our role is to disseminate PCOR. The [indiscernible] role is primarily in the production of PCOR. They have a contractor working to help them figure out what the dissemination activities should look like. We are working closely with them. It is a work in progress. I think we will get to a place that makes sense and can be clearly communicated.

I had a question -- if you were to go down the path we have critical self-examination, would you see that as an internal agency effort or something that you would turn to an external party to assist with? You may not note the answer to that yet.

Good question. In some areas, in many areas we would be looking to contract with folks to do evaluations. Having said that, contractors are still people who work for us. I don't know if you count that as internal or external. Whether we would turn to this -- and entirely external body and try to get them to do something -- I don't know who would fund them to do this. That is clearly more tricky.

I am fascinated by the healthcare innovation exchange idea. I'm not exactly sure what it is now. Trying to figure out what is next and how to get that into the market the right way. The first time. How to make sure it moves quickly to help make the changes in the system if that is where we are going. I see that as new parents -- new transparency. The tele-health industry popping up -- how do we find the things that will change the market for the good?

That's a good question. One thing that is not up here -- how to select. Mostly where we spent a fair amount of money. What came to mind.

We also have a horizon scanning contract in which a contractor scans the horizon for what is coming. With information -- potential innovations on the horizon -- makes that information available. This is used with somebody from one of the Blue Cross plans recently. He said he used it to try to figure out what he needed to be prepared for in payment policy. He knew that from the horizon scanning that the FDA was maybe going to approve something and when they approved it, they were ready to make determination policies.

The innovation exchange has been in existence for many years. It has catalogued about 800 innovations and gathers information about -- these are often delivery system reforms and profiles -- providing information on this.

Part of my interest here comes from suspect that's a pretty cool idea. On the other hand, if our job is to produce evidence and make sure that it is understood and used, very little evidence that the fact we catalogued these 800 innovation have had any effect on the adoption of innovation. There aren't some people who use this to try to figure out what is going on in care management or transitional care issues. Tele-health. But, to my understanding this is not how innovation gets disseminated and adopted.

We are working now on modifying the innovation exchange, trying to have learning collaborative pieces and see whether we can turn this into something that will actually fuel the adoption of innovation.

That is a cool website.

[laughter]

The last thing I will say is -- again, this is something that you probably know more about than I do. I think that if you do it on this pathway in it sounds like you have some resources for this which I think would be healthy, how you ask these questions will be very important. I would suggest a large qualitative component. I think it will take some peeling of the onion. If you ask most people in America or health professionals about the AHRQ score on readmissions, they would not know what to say. If you asked about project red -- they did say I get that. You fund of that work. -- You funded that work.

We are coming to the end. Before we wrap up I want to see if there are any other thoughts or comments from the room or on the phone. Regarding this meeting or things you think are pressing for us to talk about it to future meetings.

The next meeting is November 7. We will be downtown at the pump rebuilding for that session.

-- The Humphrey building for that session.

It's great that you are asking other agency directors and leaders to come and talk. It's wonderful to see everybody. Maybe as a part of the valuation there could be something about having government agencies to work together to be more synergetic. There used to be a [indiscernible] -- does this exist in the agency workgroup alignment efforts X

There are many efforts. The [indiscernible] -- QASC -- [indiscernible] and I cochaired this. This has some element folks and many other private sector folks.

There is the interagency workgroup of the national quality strategy that just met yesterday. One thing about working in government is that there are a lot of opportunities to talk with one's colleagues. Having said that, it's more -- is more coordination needed? Certainly.

I wonder if there was an evaluation roll to determine how well the coordination is happening.

Not a formal one.

One way I might recast that -- that would be a tough question to answer. A slightly different take on that -- actually an evaluation that looks at the vectors, as Harry says, through which AHRQ is having an impact? I think even a mapping of these factors would be worthwhile. It tells the AHRQ story and I have not seen the map of the vectors before. I think there is a lot more going on than we realize across the government and private sector. You sort of alluded to it whenever confronted. Just a thought for the future.

Sometimes I feel like NAC is not that helpful. We try to answer questions. It occurs to me that as you go forth you may want to put out a plan and have us respond to in and asynchronous way rather than having it come out here. Maybe looking at a plan that you thought let --.would identify a hold would be helpful.

There were some thoughts about that. It might make sense at a future forum on or on a separate call to have a structured discussion about that.

Obviously, within the limits of [indiscernible], to have some dedicated time on that.

I think we are coming to the end. Any further comments from the members? Hearing nine --

I want to make an announcement. As we think about next November -- seven of you will rotate off. Right now we have a call out for new members in the federal register notice. Please be aware of this. August 18 is the last day for people to submit applications to be on this. If you are thinking of people you think would be good to have on the board, send their names along. Contact me. Thanks.

I want to thank you again for your time and energy. And for all of you throughout the year and years. I value your input a lot. It's always hard. As Harry said, if you have other suggestions of how I could make better use of your time, I would be open to that. I found your input valuable today. The key.

Thank you.

Sake travels to those that are traveling.

-- Safe travels.

[Event concluded]