Ninth Annual Maternal and Child Health Epidemiology Conference / December 10-12, 2003

Comparison of PRAMS Self-Report of Selected Pregnancy Morbidities with Birth Certificate Records

LAURIE BAKSH:  Good afternoon.  Take a minute to acknowledge my co-authors on this, Dr. (inaudible) and (inaudible) Harrison who is our operations manager who has since left us.  Lois (inaudible) our PRAMS project director and Nan *Strater our Maternal Child Health Bureau director.  We undertook this to, sort of, get an idea of how moms are reporting their morbidities on the PRAMS survey.  And so, what we did was a little bit of background research and found that many studies have been conducted to validate the information contained on birth certificates.  Literature review shows that validation studies have been done, medical records in New York, Georgia, Florida, Alabama, Tennessee, and North Carolina. 

Some of the findings (inaudible) it all, showed a wide range of sensitivity for maternal risk factors ranging from 110% for RH sensitization down to a low of 3% for reporting of a previous pre-term infant.  And they found that the positive predictive values had also a very wide range.  Piper in their study found low sensitivity for maternal risk factors, complications of labor and delivery, abnormal conditions of the newborn, and congenital anomalies.  Study by Melissa Adams found that sensitivities for the history of poor pregnancy outcomes, such as previous pre-term infants, previous small or large for gestational age infants were low, and (inaudible) found poor accuracy for medical history, but high accuracy for method of delivery, prenatal care, and events of the labor and delivery process. 

So, I just wanted to give you a little bit of a background of how Utah does their abstraction for birth certificate records.  Pregnancy risk factors are abstracted from the medical record by birth certificate clerks that are employed by each hospital and this data comes from the prenatal care record.  Birth complications are abstracted from the labor and delivery records by birth-certificate clerk, but in Utah, the physician must have authenticated the condition on the medical record for it to be included in the birth certificate.  So either there may be symptoms that may be documented that would lead somebody to say, “Oh, well that’s pre-eclampsia,” unless the doctor has written pre-eclampsia, it is not included on the birth certificate that’s filed with the state.  Yearly quality checks are conducted by the state for each hospital.  They do a random selection and review the medical records.  So, for this study we only had one year of PRAMS data to do this with due to the change in the question. 

So, we took our 2000 PRAMS data and linked it to the birth file using the birth certificate number, and we looked at the agreement between mom’s self report of morbidity on the PRAMS survey and the equivalent category on the birth certificate.  We did a (inaudible) efficient analysis to determine the reliability between the two reports, and we used SAS and SUDAN to do the analysis.  So just a little bit of background on the--let me just raise this a little--on the KAPPA coefficient.  The KAPPA coefficient is a measure of inter rate or reliability and it measures the extent of exact agreement, adjusting for chance agreement, and so the interpretation is here.  It runs from anything less than .2 and so zero to one range, and so anything above .6 is considered good.  One thing I do want to mention is that this is not a validation study.  Without individual chart review, a validation study is not plausible and so sensitivity and specificity measures cannot be calculated. 

So the comparisons that we made were the PRAMS question on the survey that asks, “Did you have any of these problems during your pregnancy?”  And the question is a checkbox, “yes”--actually a circle “yes/no” and those that we compared was whether mom had her cervix--had to be sewn shut.  We compared that with the birth certificate risk of incompetent cervix.  High blood sugar or diabetes on the PRAMS survey is a dual check on our vita record, which is pre-existing diabetes or gestational diabetes.  Those were combined into one variable.  Water broke more than three weeks before the baby was due, same on the birth certificate, and problems with the placenta such as abruptia placenta and placenta previa.  Again, these were dual on the birth certificate, which we combined which was abruptia placenta and placenta previa. 

We also did two other variables form the PRAM survey which are two separate questions.  One is, “...whether you’ve had a previous live born infant?”  Which segues into two questions, “Did the baby born just before your new one weigh less than five pounds eight ounces at birth,” and, “was the baby just before your new one born more than three weeks before it’s due date?”  And we compared those with previous small for gestational age infant and previous pre-term infant on the birth certificate.  Those things that we had to exclude were high blood pressure, although those were included on the vital record.  The PRAMS survey includes edema with the grouping of high blood pressure and so we didn’t feel like we could include that because there’s no equivalent on the birth certificate for edema.  And we also have no variables for nausea, vomiting, or dehydration, kidney or bladder infection, or car accident on the vital record.  So, as you can see, for the six variables we looked at, we had very high agreement between the two.  However, when we looked at the KAPPA analysis, you can see that our KAPPA coefficients are quite low. 

So, we have moderate agreement for premature rupture of membrane, fair agreement for placental problems, incompetent cervix, and previous pre-term infant, and very poor agreement for previous small for gestational age infants.  So, we wanted to look at where we have disagreements.  Where are they attributable?  And what we found is that for all conditions, it’s mostly mom saying, “I had [this] problem during my pregnancy,” that was not reported on the birth certificate.  So we also wanted to look at whether there was some statistically significant differences between those women who had disagreements with the vital record, and so we looked at race and you can see the variables that we looked at--and we used (inaudible) square testing--and what we found is that there really is no pattern.  It’s wide and not the same for all variables and these are just the ones that came up statistically significant, so no pattern’s emerged for us to say, “Well, it was a specific group of women.” 

So some things to keep in mind is that there’s a really low prevalence of a certain--well of all of the conditions, I should say, less than 100 cases on the vital records for each of the variables that we looked at.  In fact, one which was previous pre-term infant, there was actually only eight reported on the vital record.  So we thought, “Well, okay, where can we go with this?”  Well women may be over reporting their symptoms but we can also say that in looking at the literature reviews that have been done that vital records data may be in error.  We can hypothesize also that more specific definitions of conditions could be beneficial to the PRAMS respondents, and also to keep in mind is that some conditions are not exactly the same between the two.  For example, all incompetent cervixes are not sewn shut and so that may be limiting the analysis as well. 

So in conclusion, we kind of--I’m nervous, sorry--we feel like continued education of providers on the importance of accurate reporting for providing sound public health data should go on.  Also, there is a motion nationally to have a national certification of birth certificate clerks, and we think that that will be beneficial in improving the quality of vital records data as all of the birth certificate clerks will have a standard training.  They will be taught how to abstract records; there’ll be a little bit more stringent requirements in terms of education and to be in these jobs.  And above all, we feel like the major limitation of the study is that we need to be repeating this study when there are multiple years of data available, and that probably the largest limitation in this study is the low numbers that we have, which is one year of data.  So--