TA WORKSHOP ON NEEDS ASSESSMENT AND STATE PERFORMANCE MEASURES
Sources of Data, Needs as Values, Need Discrepancies
January 22-23, 2004

DONNA J. PETERSEN: Alright, real quickly, and this is something that I think the Bureau has observed to their great pleasure over the years that this has become a statutory requirement as people have gotten very, very good at knowing where their data sources are, going to other agencies, linking data, people have gotten very sophisticated with this. So these aren’t things that you don’t already know. But just to put it in context, we know we get population based data from our vital records system, from the census, we know our focus is on the population, we know how to get those data, that’s pretty straight forward. We know about the surveillance systems we have in place, the surveys we conduct, either those that are nationally driven or those we do on our own.

We know within our own programs or our sister agency programs we can draw a wealth of information about clients who are served, which is not the whole population, but it still can be very valuable information, to know who is using your services in comparison to what the population data would suggest, are we missing something here? You know, how is this happening? And lastly, the data we get from the public, and I want to spend a little time talking about that in a minute, the other thing, as I said before, you're taking in information all the time. Alright? You read the newspaper, people call you on the phone, people stop you on the street, you go to meetings you hear, remember that you're often a stakeholder for someone else, right? We all sit on endless advisory committees ourselves where we give input and we take information back.

So, that kind of anecdotal information, it’s not quantifiable, it’s not a formal focus group, but it’s stuff you're taking in all the time and we can’t forget that that’s part of the mix, it’s certainly part of our thinking. The other thing, is that we have remember, that we need to, at some point in this process, assess our capacity to do whatever it is we think needs to be happening in our community. So it’s not enough to just look for needs, look for problems, or declines in health status indicators, we want to look at assets and strengths, we also want to look at our capacity, so I’m going to spend a minute talking about that.

So, I’m not going to go through all this, we know about our population based data, our surveillance systems, our programs, yada, yada, ya got all this if you want to look at it later. I want to spend a little bit more time talking about public forums, how we get qualitative data, how we get public input. I don’t have a magic answer to this. We actually had a very good discussion about this in Atlanta that I’d like to continue here, so I’d like to give you a little bit of food for thought, when I talk about public forums, you know, I’m talking about it might be a town meeting, Bill Clinton was great, he’d go around and do all these town meetings, we just had town meetings, some of you may know we just had a chemical weapons depot in Anniston Alabama and they’re now burning the chemical weapons. We’ve had a lot of town meetings about what the people in Anniston think about burning chemical weapons, in fact the joke is, if you're looking for weapons of mass destruction, they’re right in

Anniston Alabama, you didn’t have to go so far to look for them. So there are town meetings. There are public hearings, do you use this mechanism? They're very formal, right, the panel of, what do you call them, hearers, and people come in and it’s very formal and you get 3 minutes and you testify and you move on. There is no dialogue, you're hearing and if someone wants to do a change in a zone there’s a hearing. Eminent domain there might be a public hearing. We want to put a group home for prisoner’s early release, you better believe people are going to show up at a public hearing, give their input on why they don’t want that in their neighborhood. So, you know, that’s not the right mechanism for this but it might be if something comes up controversial enough that you want to do something that forum, formal. Many of us use, we use focus groups as a way to, in a more formal way, gather some qualitative information from a group of people that we have selected based on certain characteristics to try to inform what we’re doing.

And we can use focus groups anywhere along this process. We can use it up front to say ‘tell us what do you think the needs are of women in this community’, we can use it up front to say ‘alright, here’s what our data tells us about the health of children, help me understand that’, we can use it later when we say ‘these are what we believe are our priorities, are they or aren’t they?’ We can use it later to say ‘ what would you think about group homes for prisoners in our communities?’ And later, you know, how do you feel about this particular strategy? You can use it anywhere along the way. They’re a very powerful tool, and as I say, we get lots of anecdotal data. What we talked about in Atlanta, clearly stakeholder involvement is a form of providing public input. That’s one mechanism for trying to get public input And when I’m talking public input now, I’m not talking about at the very end where you’re supposed to tell us how you got input on the block grant application.

That’s part of the block grant application, you have to say how you got input. And we all struggle with this, you know, no one wants to read the bloody thing, I don’t want to read it, the people who wrote it don’t want to read it. Okay, so you know, that’s too late then. You know, we’re not going to get people to read this thing but we can get people to comment on our process all the way along. We can document that. So stakeholders, stakeholder involvement all along the process can help you do that. Focus groups, we talked about but just really want to hear from you, anybody that wants to share with us. How have you gone about eliciting public input outside of a stakeholder group, outside of an advisory committee. What have people done? And I’ll share with you some things we learned in Atlanta. Does anybody have a good story about a way in which they got some good public input?

UNKNOWN SPEAKER: Inaudible.

DONNA J. PETERSEN: Connecticut, street intercepts. Tell us more.

UNKNOWN SPEAKER: Inaudible.

DONNA J. PETERSEN: So using an ethnographic method, one ethnographic method, street intercepts, she called it, where you basically just have trained people out there stopping people on the street and asking them some critical questions. You did this in oral health; you did this in trying to understand Latina adolescent pregnancy. Interesting. She said when you don’t have the money to do focus groups, focus groups are not free, they can be quick but not free, you have to get people in, you have to feed them, you’ve got to record a useful method but here’s another one. They do that first. They do that to get some immediate quick, you know, what’s going on in the minds of the public and then turn that into focus groups so again, in this sort of iterative process, it’s a good way to get quick “what do people think about all this”. Certainly this is done in the media all the time, marketers, business people, as you say. Other ideas? That’s a good one. Bonnie. Please talk into the microphone. This is too good. We want to capture it.

BONNIE: This is a question. I wonder if anyone has started using the Internet, or some kind of web assisted technology, because I’d be really interested.

DAVID: Not actually the Internet, but for street intercept interviews, palm surveys actually work pretty well for some types of surveys and they’re relatively easy to get the software and do the programming yourself once your survey is developed. It’s a tool that works well.

DEBRA: Actually I sit in on the Infertility Prevention Project and Johns Hopkins in Baltimore City has a plan to use the Internet for recruitment to get some focus groups going for use of their clinics with respect to Chlamydia. It hasn’t actually come to fruition yet but they're moving forward with it, so we might be able to take a look at that, Bonnie.

MICHELLE: We use a public website to advertise our public forums that are sponsored by our advisory council and also, at that same time, to let people know that they can either call us, write us letters, or send to this email address with their comments that are on this application and any other issues that they have about MCH.

DONNA: Is that on there all the time, that open invitation is on your website all the time?

MICHELLE: Yes.

DONNA: Okay, Cathy?

CATHY: Again, we haven’t tried this yet, but what we have been talking last time we did this, we have a written survey kind of thing that we handed out to various groups like parents of children with special needs, what are your top needs, what would you like to see different, happen differently in 5 years. What we’ve talked about doing this time is making those kinds of tools available on our website, then doing publicity with various groups, so if schools wanted to use them, that they can always go find the tool when they are ready they can make copies of it and hand it out at something so it isn’t so much our having to be at a meeting, or mailing something out. But we haven’t tried it yet, so I don’t know if it will work.

UNKNOWN SPEAKER: How many states have utilized the public library? That’s an area I think you may need to explore, and in Connecticut that’s what we’re trying to do with our public library because it’s in every local community and most of them do have community activities and web base that parents of low income families, I know that people that we are looking at go to the public library with their kids to do homework, and they do parent education and web base. So we’re partnering right now with our state librarian to utilize the system catered for parents in the local community statewide called greatkids.org to use it as an access point to get public input and also involve people and announcements, so you may explore your public library.

DONNA: Neat. William, please.

WILLIAM: We utilize I guess what you would call a conference model, because we have the annual conference which annually attracts about 900 people and this year we’re trying to strengthen it by having a focus session on the MCH block grant process and in fact, we are utilizing resources from participants who are here now and so we hope to strengthen that to get additional input as well as, we started last year a partnership round table, bring together all our partners, our MCH partners, and that’s a way we use to get some input.

DONNA: Good. Terrific. Some other things we talked about in Atlanta, one, people, people were very cautious about these open public forums, that people had had really bad experiences where it just turned into chaos, someone had made the point earlier about good facilitation, and good facilitation in something like that is absolutely critical, you got to have somebody who can manage that kind of process, keep it from completely erupting and then that led to somebody sharing this, which I never heard it called this, they called it a carousel model where if you have a conference, or you know, you don’t call it a public hearing, bringing people together and then, in essence, having tables where there’s an issue at each table and you're given so many minutes to go sit at that table and with whoever’s sitting there, whatever you want to say about infertility. Bingo. Next table, whatever you want to say about adolescent pregnancy. Bing. Next table. Whatever you want to say about transition for children with special needs.

And some people will chose not to go to every table because they have nothing to say about a particular issue, others have something to say about every issue, they go to every table, but it was a way to control the process without controlling the dialogue. I thought it was a really neat idea. And they said they’d done this, this was, was it Kentucky? I don’t remember, but that they’ve done this in professional conferences as a way to get input, they’ve done it in larger community forums where they bring people in to hear a speaker that’s of interest to people and then get them engaged in this kind of process, so I thought that that was really neat. I know that back when I was in Minnesota we did something like this where we needed really critical input on some of the health reform planning that was going on, and we brought people in and we, in essence, segmented them into 3 groups, women and infants, children, children with special healthcare needs and worked them through a process over the course of 6 or 7 hours and then came out.

The other thing that, which I’ve already kind of alluded to is this idea that you don’t have to always be the convener, you can go to where people are already convened. And if you can get on the agenda of somebody else’s meeting and you know, maybe just for a few minutes, as Cathy says, as others have said, if you’ve got a quick survey instrument that you want to fill out quickly for you, if you want to ask them a few key questions, get some dialogue, most people will let you do that. You know, there’s always 30-40 minutes on an agenda they can spare for you. So, again, thinking about your partners, and they're already part of your roundtable or your stakeholder group that’s going to be an easier thing for you to have access to so you can use those networks.

Well, well! I’m going to add your suggestions to the list, if you don’t mind, we’ll keep building it, again, we’ll come back to these things and share some of these things on the web site. Capacity assessment, as I said, really important not to forget this. Somewhere along the line, and part of this you do it anyway, you do it, part of, perhaps the reporting you have to do, but it’s really important to think about what services are already out there, what’s already available, maybe being under utilized, maybe with slight tweaking could be more effective given the needs you’ve identified. Who’s out there? What kind of services are out there? What kind of providers geographically? Are they in areas where people can get to them? Are they accessible? Is there continuity of services across different providers? And again, what you're trying to get at is are we fully utilizing that which we already have out there?

Are we optimizing the resources we already have? Again, they’re not always yours. You know it’s not just what you do. Are there community health centers out there, what are they doing? Where are the tertiary facilities, where are the pharmacies? Where are, all the kinds of outlets that you can think about? And clearly, at the state level you know you’re looking at that broad capacity, what programs do I have in place, who’s eligible for those programs? Is that something where I might be able to maneuver? Is there access to these programs, you know, how are my providers and facilities distributed? And, you know, those are things that we sort of do routinely. We know where there’s no pediatrician in the state, we know where that pocket is, we know where people can’t get to that hospital when the weather’s bad. Those are things, and just keeping that in mind. At the local level you can be much more specific. Who’s out there providing what service to who, when. All of those things. And so, again, depending on how you do this what ability you have to engage some local partners, that can be really helpful to you to figure out what’s going on, what might be missing.

I did some work in Montana years ago, anyone here from that part of the country originally? I had never been out there, I’m a New Yorker, and they would talk about being hours from anything and there is no tertiary facility anywhere in the state of Montana, you have to go to Salt Lake City. And they thought nothing; their idea of community was so different from mine. They would talk about living in one county, the county road being 2 and a half miles from their house, the driveway was 2 and a half miles long, the cattle ranches, their children would get picked up by the bus, ride an hour and a half to a different county, because there are no schools in their county, there aren’t enough kids, and they went to church in yet a third county, because that’s where the Lutheran church was, but their husband’s favorite hardware store was in yet a fourth county so that’s where they go to get stuff, and we oh, we love going to the rodeo and that’s all the way in another state, that was community to them. Wherever they worked, lived, played, interacted with people, that was community.

So really it’s just an interesting view of the world and so looking at capacity there is going to be very different from Connecticut where you have a densely populated community. So just something to think about. So how you look at that, how they would look at that versus you is going to be different. So obviously within your own circumstances, I want to go back though to this one slide just to remind us that resource inventories are really about capacity. They're not about need. Just because something’s out there doesn’t mean, don’t assume that means anybody needs what they're providing, okay? It’s just what’s out there. It doesn’t say anything about the quality of what’s out there, you're just documenting that it’s there, and it doesn’t really assess the effectiveness of that particular service within the overall system. It’s truly just a, an enumeration of your capacity.

DONNA J. PETERSEN: Alright. Now, we’ve done great work, we have stakeholder committees, we have gathered data. We’ve got our data folks, you know, churning this stuff, we’re out there doing street intercepts and focus groups and we’re holding these conferences and we’re doing capacity assessment, we’re going to have a huge list of possible issues, right? We’re unlikely to come out of here with 2 or 3 things. We’re likely to come out of here with 20, 30, 40, 100, you know, and all over the map. Right? We’re going to have just this huge array of things that we might address.

So how do we sort them out? First, we have to remember what needs are. In this spirit that we’re talking today, about community and engagement and the population, we have to remember what needs are. Needs aren’t statistical indicators. They really are expressions of what people value. For something to be a need, somebody has to care about it. It has to be something we value. And we may have opposing views within our community about what we value and don’t value. These are expressions of values. And the way this plays out, is it’s clear to you from the data what the problem is. Nobody cares about that. You go out into the community, not an issue. They care about something very different, because that’s what they value. And part of it, value, if you think about it, you know, it’s what we care about it’s what we value, it’s what’s important to us, it’s also sort of in the scheme of things, you know, if I don’t have decent housing or a job or good transportation it’s hard for me to get exorcised about whether my child’s getting preventive healthcare visits. You know, so, you know you always have to be kind of balancing this stuff.

What do people care about? And so once we determine what we value, what we care about, then we have to decide, okay, well, how far are we off the mark? So what we’re really looking at here is discrepancies in what we value, what exists, and what we would like. Then we have a need. Needs are value judgments that suggest that problems exist for specific population groups or in specific communities. So there’re going to vary, they're open to disagreement and debate. But for them to be useful to you in your planning there has to be some agreement that they're real, and that they're worth investing some time and energy and resources in.

If you come out, well, I’ll tell this story because it’s a great story, some of the Maryland people might even remember this. Governor, there was money left, this is how long ago this was, there was money left in the preventive services block and the MCH block in this particular year when I was working at the Maryland health department, can you believe it, there was money left. The governor at that time wanted to target that money towards some children’s initiatives, he was big on children’s stuff, and that was consistent with both the block grants, so that worked, but he wanted to announce it in the state of the State address which meant we couldn’t talk to anybody about it, right? Because it had to be a big surprise when he announced it in the state of the State address.

So what did we do? We went to the data, right? Because that’s what we would do, we’re good public health professionals and we went to the data and we looked at all kinds of possibilities, and we had a whole package of things, but within that package of things, we targeted the Cherry Hill community of Baltimore which is a very, probably still, I can’t believe it’s changed, has it changed? Cherry Hill is still, you know, it’s off the charts, every indicator you look at, Cherry Hill comes up over and over again. We were really worried about the youth in Cherry Hill, we were going to do some after school stuff to keep them off the streets and we were going to work with the schools and we just had all this great ideas. We were going to get parents involved, wonderful stuff, we were so excited, and the governor announces this, and because we’re not dopes we knew we’d have to immediately, once this stuff was announced, we’d have to get out there quickly and say “look, these are just our preliminary ideas, we really want to hear from you”. So I trot off to Cherry Hill, this was my baby, I go to Cherry Hill, we had a community meeting, there were lots of people there, I was really impressed that so many people were excited about my program for youth.

It turned out what they were excited about there was actually somebody there from the health department who actually came to Cherry Hill to talk to them. And what they wanted to talk about was rats. After school photography classes? You’ve got to be kidding. We have a serious rat infestation problem in this community and no one has ever come down here to ask us about it. The whole thing blew up. I mean I went back to the health department with my little tail between my legs; I was mortified and thought my life was over. The head of the preventive medicine administration at that time wasn’t there and actually called me at home that night because he knew I was just crushed, went back to the health department, went to the rat control guys who were buried by the labs, is that still over there?

The dark hallway, you know they're buried back there, the rat guys, and they said, “well darling we could have told you that”. So even though I thought I’d done my homework in the agency, I missed this entirely. So the point here is, you know, what did they value, what was important to them at that moment? Of course they value their children, that wasn’t the issue, but the point is if I can’t even live in an environment where I’m not confronted with rats all the time, how can I think about anything else? I learned that one the hard way. I don’t think I’ve ever been back to Cherry Hill since. Send somebody else.

So how do we figure this out, how do we figure what this discrepancy is? How do we know what it is we want? We have to be able to articulate that. What is it that we want? Cherry Hill wanted no more rats. They could articulate that very clearly, and in fact, you know this thing blew up so badly that the mayor had to go live on the news that night, and as luck would have it, he was standing in a vacant lot, 5 o’clock, 6 o’clock news, it wasn’t even nighttime, this rat the size of my dog runs across his feet like on cue, I’ve always suspected somebody let it out of a cage. So, we can do this in a couple ways. One is, we can look to what experts say. And this is what we often do. We say “alright, okay, I know what the situation is here, but so what? Is that good or bad or what?” So we look to other people’s assessment of what would be the standard, so if we’re in a state, we look to the national rate.

Right? We do this all the time. Look, we are Maryland, and we are below the national average on this indicator and we want to get up to that level. That’s how we set the discrepancy. We might say that well, what we want here is an ideal. We want no rats. We want no infant deaths. We might not be very realistic, but it’s a way we might do it. We might say we don’t ever want to be any worse than we were in 1990, you know that was the lowest, and we never want to be, that was the high, we can set something. We can look at healthy people 2010. Somebody has set out objectives for us. And, of course, if we’re Vermont or some of these states that because of their great leadership in recent years, have done well in a lot of these indicators and there’s nothing left to compare themselves to, well, you know there’s always Japan or Sweden, you can find some country with an infant mortality rate worse than yours, you know there’s always someplace you can find a discrepancy. We look to how experts define this.

That’s one way we do it. The other way we do it, is what we call, sort of expected or desired which is where we ask the public to tell us. Now, they often go to either the ideal, no rats, or something more like the optimal, like the Healthy People 2010. They can be reasonable, they can understand that if Japan’s infant mortality rate is at 5, it’s unlikely that we’re ever going to get below that, so you can work with them to set those standards. But you let them tell you what the expectation is. And sometimes you find, you know, people will say, “we have this dreadful need look at these numbers” and then when you look at them later you say I understand people value this, but we’re really not doing that badly compared to other people, it might not be acceptable to us, so we might say we still would like to be not that bad, we’d like to improve on that. Let people tell you where they would be comfortable.

The other way we do this is in the language of supply and demand, but you’ve got to be careful about this. Like I said before, just because people are demanding something we often then equate that to need and you’ll see people say well, we know we need this because 11,000 people utilized this service last year and we had a waiting list of 4,000 therefore there’s a need. Well, you don’t know that for sure, unless you’ve truly documented the need, all you're documenting there is the demand, and it’s the demand for what you provide. They might actually want something else. You know, when my kids force me to go to McDonald’s, I do not want to be there, I still buy something but I didn’t really want it, and I know I didn’t need it, especially those French fries. So, what we “demand” in that supply and demand terminology might not really be what we need, so we have to be careful of that. It’s also back to not doing needs assessments to justify our current efforts.

You don’t want to go back and say ‘well, clearly we need these services because look at how many people come and demand them.’ That’s not quite what we mean when we say doing good population based community driven needs assessments. But the point of this is, it’s really important to figure out where people are coming from in all this. What do they really care about and what do they want to see? We’re talking change here, and if you're going to change what are they willing to accept in the way of change? That’s why you have to get at this, what is it we’re trying to get to? The other reason you have to have these conversations is that ultimately, we’re going to touch on it later today and you're going to spend all of tomorrow doing it, you’ve got to set some targets for yourself. So, you really need to know what is it people are expecting or stating as an expert viewpoint what it is we want to do.