DONNA J. PETERSEN: What I have put at most of everyone’s seat--I ran out of copies--is just a little description of what you would now do with this Q-Sort data, because we don’t have time. You’re too big for us to go through it right now, but I want to tell you what you do with this. So you have everybody’s sort. They’ve filled in their grid. Then, you or somebody--and you can actually do this on site, I’ve seen people do this--you enter it into a spreadsheet. And what you’re basically doing is, you know, need number one, access to care: how many people rated that a one, a two, a three, a four, five, six, seven, eight, nine. You assign a score. And so you enter all this data, and then you can derive mean scores for each need, okay? You also derive the standard deviation, and the reason that’s important is it tells you how much consistency or inconsistency there is in the voting. The reason that’s important is if something rates a fairly high score and has a small standard deviation, you’re done with it, okay? You know that there’s general agreement that that’s a high-priority issue. Same with the low end; something comes up low and the standard deviation is tight, you know that there’s fairly general agreement that people don’t think access to care is an issue.
The ones you have to discuss are the ones where the standard deviation is large, because it means you’ve got some people that feel strongly that it’s a high priority and others that feel less strongly about that. Those are the ones you need to discuss. So it’s a way of focusing your energy and your time and your discussion. It’s also one more way that the group speaks, right? You know, and it might not be to your liking, either; but if this is where it comes down, this is where it comes down. You’d never take anything off the list. You don’t have to. But now, you have a sense of where at least this group is. It only works with larger groups. If you have too few numbers, you don’t have enough data to have meaningful standard deviations. So if you’re using it with a small group, it’s still a fun exercise. And with a small group, I could actually draw that pyramid, and we could go through and--and get a sense of who was interested in what. So you can still use it and have some fun with it at a point in conversation. When a lot of people say that unintended pregnancy is the top issue, and somebody else says, “Well, I didn’t see that as an issue at all,” you can have some dialog about why that is an issue or not. Okay? So it’s something that you might want to use. I’ve used it a lot, and it’s fun. People enjoy doing it. They like spreading those cards out. And I’ve also done this in small groups, so the group has to come to some consensus, which is what we’re going to do now.
This is the neat thing about a Q-Sort is you let each individual come up with their own scores, but that’s not reality because at some point, we’re going to have to come to some consensus about these priorities. So the next exercise we’re going to do so that you don’t fall asleep after lunch--who had ice cream, by the way? See, we’re in trouble. Same data, same set of needs. We’re going to put you in some hopefully different groups, force you to come to consensus on the top three. Following the exercise page, you’ll see exercise two. The page behind that are those same needs, but now I’ve given you the data. It’s right out of Healthy People 2010, same list of needs in alphabetical order, and I’ve told you what they report as the baseline and what they report as the objective. And in some cases, we don’t have any data because it’s developmental. Okay? You know what you came up with when you did the Q-Sort, but now I want you to just take a couple of minutes, look at the data, and see if that changes your mind. Just take a few minutes, and then I’m going to put you back into some groups and we’re going to come to agreement on our top three.
A couple of these groups got done quicker than others. Some of you, one of
you. David, your group? You didn’t? Okay. Who did it? Five? Just tell
us real quick, what were they?
UNIDENTIFIED SPEAKER: They were obesity in children, access
to care, low birth weight.
DONNA J. PETERSEN: I’m going to put these on flip chart.
Obesity in children, access to care, low birth weight. Another group that had
an easier time of it?
UNIDENTIFIED SPEAKER: Access, obesity, unintended pregnancy.
DONNA J. PETERSEN: Access to care, obesity in children, unintended
pregnancy. David, your group? Who’s reporting from that group? Back there.
UNIDENTIFIED SPEAKER: Number one (inaudible) access to care.
Then, it was special needs and prenatal care.
DONNA J. PETERSEN: All right, access to care, children with
special needs.
UNIDENTIFIED SPEAKER: (Inaudible).
DONNA J. PETERSEN: See?
UNIDENTIFIED SPEAKER: (Inaudible).
DONNA J. PETERSEN: Minority report from group five?
UNIDENTIFIED SPEAKER: Access to care (inaudible).
DONNA J. PETERSEN: Okay. All right here. What was it? It was
health insurance? Children with special needs. This group up front, Maureen.
UNIDENTIFIED SPEAKER: Access to care, unintended pregnancy,
adolescent (inaudible).
DONNA J. PETERSEN: Not this group but one more. Who else? Who
am I missing? Tony?
UNIDENTIFIED SPEAKER: This was very difficult for us. I’d
like to preface it that we didn’t trust (inaudible). And they only agreed
on one. Based purely on the data that was provided here, children with special
healthcare needs was our top priority.
DONNA J. PETERSEN: Okay, am I missing another group besides
this last one?
UNIDENTIFIED SPEAKER: (Inaudible).
DONNA J. PETERSEN: Okay, and then this group in the back. Tell
us what you were doing back there.
UNIDENTIFIED SPEAKER: Well, we were trying to develop some
criteria for selecting (inaudible) and came up with those.
DONNA J. PETERSEN: These down here? Nutritional intake, tobacco,
unintended pregnancies, STDs, and asthma? That’s as far as you got. Okay,
so this was hard.
UNIDENTIFIED SPEAKER: (Inaudible).
DONNA J. PETERSEN: It depends on who was in the group.
UNIDENTIFIED SPEAKER: Who you invite (inaudible).
DONNA J. PETERSEN: Yeah, who do you invite to the party? Now,
you all are presumably of somewhat like minds.
UNIDENTIFIED SPEAKER: Well, it (inaudible) became apparent
also was the issue between those who wanted to continue to vote emotionally
and those who wanted to consider data.
DONNA J. PETERSEN: No, good. I’m very glad you pointed
that out, because you are not going to have the benefit of having a room full
of people who are ostensibly like you, and you’re going to have people
that want to go with the data, people that don’t care what the data say.
Now, how many people changed their minds from what you did in the Q-Sort to
when you looked at the data? Okay, so it did influence you. Good or bad.
UNIDENTIFIED SPEAKER: (Inaudible). I thought it said STDs
other than (inaudible). Yeah, so (inaudible).
DONNA J. PETERSEN: I’m sorry. So there’s a couple
messages here. One is this is not an easy thing to do, and obviously you would
have given it more time; and in fact, you might give it multiple iterations
if you were really doing this. And you might even consider using a Delphi approach
where you don’t even bring people together. You do some of this before
you get them together. That’s number one. Number two is where do these
data come from, anyway? What do these things mean? This was right out of Healthy
People 2010. I didn’t make this up, you know.
UNIDENTIFIED SPEAKER: It was creepy.
DONNA J. PETERSEN: What? It was creepy. Okay, you know, and
as I said before, that was a political exercise in and of itself. And don’t
be fooled. You know, all you MCH epidemiologists in the room, purists that you
are, you know, how you present these data, how you define terms, you know, that
all can influence people whether you intend it to or not. People are going to
interpret these things. They’re going to say, “Oh, I didn’t
understand that that’s what that meant. Oh, I didn’t realize,”
or, “What does this mean? What does that mean?” So, you know, maybe
that’s part of the training that we talked about earlier, that you’ve
got to spend some time having people become comfortable with these data, and
you’ve got to be willing to kind of have a little give and take to say,
“All right,” you know, “Maybe I can make that a little easier
for you to understand or break it down by population group or something.”
Bernadette and then Kathy.
BERNADETTE: (Inaudible) access to care (inaudible) individual
(inaudible) indicated why they wanted access to care. (Inaudible) came in that
we all were repeating that (inaudible) back the access to care that we have
already achieved would collapse. And so everyone brought it in as a first priority,
whether it be (inaudible) actually still believe that it would be possible and
necessary to (inaudible). My question to you is oftentimes; we make those decisions
that we’ll still keep something a priority, even though maybe the data
reflects (inaudible) because we’ve reached such a high plateau. We’re
all afraid if we don’t list it as a priority, it will be presumed that
they can (inaudible). Is it validity (inaudible)?
DONNA J. PETERSEN: Yes. Let me recap to make sure everybody
heard it and for our guests out in TV land. The issue was she said that a number
of people spoke up right up front talking about access to care and why they
thought access to care should be the top priority. And the issue was we’ve
achieved a very high level of access to care, but we’re worried about
with dwindling resources and perhaps changing priorities, we don’t want
to lose ground in that. So the people want to keep it as priority because you
don’t want to have it then diminish. Yeah, it's a very legitimate issue,
and I’ll respond in two ways, and then I’ll let other people chime
in here. One is, I said this earlier, that you might have an “A”
list, a B list, and a C list; and that there are those things, I said as an
example things that you’re mandated to do, but you could also put on that
list things where you’ve achieved a level that you want to maintain. You
don’t expect to get much higher, but you certainly don’t want to
get any worse. You don’t want to get any lower. You know, should that
take a priority slot away from something potentially emerging? Well, you know,
that’s part of how do things work in your state and what are the costs
and the benefits of doing one or the other? But if you leave it on your B-list,
it doesn’t go away.
The other thing, when you talk about performance measurement tomorrow, you’re
all reporting on those national performance measures whether they’re your
priorities or not, correct? Much to your chagrin. You then have a set of state
performance measures, and it’s a small list. Probably, in reality, you’d
have a much bigger list. And probably, in reality, you do have a bigger list
somewhere else. The whole key to performance measurement is it ought to be something
you expect to change. The whole word “performance” implies you’re
doing something. You might consider not selecting for a performance measure
something that you’re just maintaining, but you’re going to keep
monitoring that. That’s clearly going to be on a list of things that you
pay attention to because if it does start getting worse, than it might need
to flip right back up into a priority and a performance measure again. Is that
making sense? You know, you don’t take things off the list. All we’re
doing now is working our way toward some priorities, things that we want to
be sure we focus on. If you believe in your heart of hearts that there’s
threats to this and you want to make sure it doesn’t get worse, then you
might choose to have it be a priority. Kathy and then Mary.
KATHY: That’s what I was going to say. I think particularly
in this environment, and we’ve had some. Again, Massachusetts has really
good infant mortality rates, whatever, you know, some things. But we consider
them still in jeopardy if one doesn’t. So in some cases, we’re saying
we’re just hoping actually by setting something that’s a relevant
(inaudible) stable achievement, that’s going to take some effort. In other
words, the grain--
DONNA J. PETERSEN: Right.
KATHY: --is going against—
DONNA J. PETERSEN: Right.
KATHY: --not having an increasing in teen pregnancy rate, for
example, or something else. And then, it really depends. You have to explain
that; you have to understand it; and if you only have 10 to choose from, you
may make a slightly different choice. I was also going to say earlier, one of
the things in our group, which I don’t think maybe would be much of a
problem in the States but it might, people had come to their first list thinking,
from our own states, for example, in some cases. We were then looking at national
data, which, in fact, well, our data don’t even look like that.
DONNA J. PETERSEN: Right, and a couple people made that point.
KATHY: So it’s like well, which way should we be setting
only talking national priorities there, as opposed to what do I really think
would be the top ones.
DONNA J. PETERSEN: Sure. And within your state, obviously you’d
use your own data; but that’s not to say that you’re not going to
have somebody from a local community who only sees the world that way, okay?
Mary? The comment was that there are things that you might be doing very well
but that you don’t want to, you know, you’re worried that because
of fiscal forces and other competing priorities that you might slip, and that
it’s going to take effort to keep those things up where they are. I mean
the best example of all is tuberculosis. I mean that’s the best example,
and most everybody’s going to know what you’re talking about when
you say that. You know, we thought we’d solved that problem. We took the
money away. We went away, and lo and behold, it hadn’t gone away. It’s
still with us. The only thing that went away with us. Mary, you had a comment
in the back.
MARY: I think I may be saying the same thing. I think personally
that it’s quite legitimate to have a (inaudible) sustain something or
to maintain something.
DONNA J. PETERSEN: Please, it’ll make them happy.
MARY: --sustain or maintain, particularly in a situation of
level or diminishing resources. Now, it might not be the positive spin that,
you know, is being pushed; but still, I would argue it could be very good planning.
DONNA J. PETERSEN: I think all these things are true. Her point
was, you know, that again, it’s going to take effort to sustain some of
these things. Maybe it’s not the new, trendy, you know, exciting thing,
but what do you have, 10 priorities? So, you know, balance some of this. Maybe
this is so important to so many people that nobody wants, you know, we’re
not at 100 percent yet, so there’s room for improvement. We don’t
want it to get any worse. So many other things are dependent on this thing that
we do leave it up there as a priority, and we explain why, as people have said.
Okay? Does that make sense? I know what Cassie is standing over here for, I
think. Go ahead. Then, I’ll (inaudible).
SALLY: Is this on? It’s on.
DONNA J. PETERSEN: No. It’s being recorded, that’s
Sally.
SALLY: And then, what we were talking about in our group was both a needs and a strengths assessment. And if we had rated something real low, does that mean--turn it around--does that mean it’s high on the strengths list, or the assets list, or does that mean for another reason we rated it low? So we went back and forth with taking this and turning it around, and, you know, you could make it 3-D if you wanted to--
DONNA J. PETERSEN: Absolutely.
SALLY: --to pull in both the strengths and assets assessment
exercise for developing consensus.
DONNA J. PETERSEN: Absolutely, and this is also why this takes
so long. You know, you can’t do this in one meeting. You know, all the
things we talked about earlier, building trust, making sure everybody’s
up to the same level, kind of hearing people out, you know, this all takes a
long time and we’re all going to interpret things differently. And you’re
absolutely right. You know, other people might have rated access low because
they were making assumptions, “Okay, it’s high. It’s going
to stay high. I know we’re going to devote energy to that. I’m not
going to worry about that,” where other people said, “I am really
worried that we’re going to lose ground here.” So you’ve got
to have time to discuss all this.
You’ve got to be able to explain why you did something the way you did.
And again, that’s what’s nice about that Q-Sort. I mean that’s
an example, perhaps, where this might have shown up as 1.7 with a very narrow
standard deviation. We know everyone’s worried about it or cares about
it or wants it on the list. It might have had a wider standard deviation, and
then you could talk about that. Well, why did some people rate this lower and
people rate it higher? But coming to consensus is really, really difficult.
Speaking of consensus, we had some people join us after we did introductions
this morning, and I want to make sure we know who those folks are that have
joined us. I’m not sure who you are because we moved you around. But Joyce,
I think you came in after we started.
JOYCE BROOKS: Yes, good afternoon, everyone. I’m Joyce
Brooks, and I’m with the D.C. Department of Health and under children
with special healthcare needs, also.
DONNA J. PETERSEN: Yvette, did you come in late?
YVETTE: Yes. I’m Yvette (inaudible) Center of Maternal
Child Health (inaudible) Department of Health (inaudible).
DONNA J. PETERSEN: Anybody? Who else came in later? William.
WILLIAM: William (inaudible) Department of Health and Maternal
Child Health Officer.
DONNA J. PETERSEN: What state?
WILLIAM: D.C.
DONNA J. PETERSEN: District of Columbia. Welcome. Was there
somebody else from Connecticut came in? No, you’re all here. Okay, welcome.
Just wanted to make sure we know who’s here. All right. So we struggled
with this. You know, this group went right to “We’ve got to have
some criteria here, because otherwise we’re just fighting.” There’s
lots of tools
you can use.