Bollettino epidemiologico nazionaleSIMI and SEIEVA: A comparison of two surveillance systems for Hepatitis A in Puglia
Pietro Luigi Lopalco, Rosa
Prato, Caterina Rizzo, Cinzia Germinario, Michele Quarto
Institute of Hygiene – University of Foggia, Puglia
Regional Epidemiologic Observatory, Puglia
DIMIMP – Hygiene Unit, University of Bari, Puglia
Hepatitis A surveillance in Italy is part of the mandatory infectious disease reporting system. In the Puglia region in southern Italy, which has a population of approximately 4 million inhabitants, this information system was computerized in 1996 as part of a national project known as Sistema Informatizzato delle Malattie Infettive (SIMI; Computerized Infectious Disease System) [1]. Beginning in 1997, Puglia also has participated in the Sistema Epidemiologico Integrato per le Epatiti Virali Acute (SEIEVA; Integrated Epidemiologic System for Acute Viral Hepatitides) [2], a special surveillance program in which information on clinical characteristics and risk factors is collected by members of the Public Health Service Units of the Local Health Agencies (AUSL) that participate in the system. The AUSLs transmit information on the number of cases to a regional coordinating unit on a weekly basis, and on a monthly basis, they submit the completed questionnaires on each notified case.
SIMI is based on the legally mandated surveillance system and has been in operation several years. SEIEVA, which is a more complex system, provides more timely and detailed information on risk factors. Under ideal circumstances, all the cases should be notified to both systems. However, since both are passive surveillance systems, both are likely to be subject to a certain amount of under-reporting of cases. An investigation of the level of dual notification was considered of use in evaluating the quality and completeness of the two systems.
To estimate total cases, dual notification, and the sensitivity and completeness of SIMI and SEIEVA, the records of cases of hepatitis A notified to SEIEVA and SIMI during 1997-1999 in Puglia were linked. Because SEIEVA was based on anonymous reporting, linkage between the two systems, linkage was based on initials of the individual’s first and last names, the age in years, sex, and month of notification (for SIMI, this was the month in which the case had been notified by the AUSL; for SEIEVA, it was the month in which the interview of the case was conducted).
Although data were provided by two different systems, the information sources were interdependent, making it impossible to use mark-capture-recapture techniques [3] to evaluate the total number of cases. Instead, to determine the total number of cases, we determined the number of cases identified by both systems and subtracted this value from the sum of cases notified to each of the two systems. For both SIMI and SEIEVA, sensitivity was calculated as the percent of cases reported to both divided by the total cases reported to each. Because one of the 12 AUSLs in Puglia (AUSL Lecce1) did not participate in SEIEVA until 1998, both SIMI and SEIEVA data from this AUSL were not considered in the analysis of 1997 data.
During the three study years, 6768 cases were notified to SIMI and 4170 to SEIEVA. Most of the cases were reported in 1997 and were part of a prolonged epidemic that had begun in 1996. In 1998, the number of notifications decreased to less than 1000, and subsequently were reduced by half in 1999 [4].
A total of 2455 cases were reported by both systems, representing 36.3% of the total cases reported to SIMI and 58.9% of those reported to SEIEVA. The total number of cases reported to both systems was 8483 (Table). The sensitivity of the SIMI system for Hepatitis A was 79.8%, while for SEIEVA, the corresponding value was 49.2%.
During each of the three years, sensitivity was considerably higher for SIMI than for SEIEVA. No apparent trend was evident in sensitivity of the two systems during the study period, although in 1999 the sensitivity of SIMI was higher than in 1997 and 1998 and that of SEIEVA was lower from 1999 than in the previous two years.
During the three years, the overlap in reporting between the two was significantly different in the five provinces of Puglia included in the study (p<0.01), ranging from 31.6% in Bari to 84.1% in Foggia for the SIMI system and from 49.6% in Brindisi to 79.5% in Foggia for SEIEVA.
The most important result of this study was not so much the high number of SIMI notifications that were not present in the SEIEVA system but the finding that there were cases notified to SEIEVA that had been not notified to SIMI. This is surprising because the SEIEVA system is more labor-intense and requires an interview of each patient. In fact, during the three study years, a total of 1715 patients included in SEIEVA were not notified to SIMI. This may be due to the belief by health care workers that their reporting obligations had been met by completing the questionnaire and interview required by SEIEVA and that it was therefore not necessary to report to SIMI.
This evaluation enabled us to identify organizational deficits in the surveillance systems that can be corrected with better training and by involving those performing the surveillance in evaluation activities. Among the corrective measures, feedback of surveillance data may serve to increase the motivation of those involved and correct the identified deficits. On the basis of these results, a meeting was called with those responsible for the two systems in their respective AUSLs and corrective measures were undertaken. Using data from 2000, it will be possible to determine whether levels of agreement between the two systems have improved.
Commentary
Editorial Committee, with assistance from Alfonso Mele and Stefania Salmaso, Laboratory of Epidemiology and Biostatistics, Istituto Superiore di Sanità, Rome
Rarely are evaluations of surveillance systems performed, and even more rarely, the results of the evaluation are used to correct the deficits identified. Reasons include the problems of defining the primary objectives of the surveillance system; the lack of flexibility of most existing surveillance systems, and the difficulty of identifying those willing to assume “ownership” of the system. This comparison between SIMI e SEIEVA is useful because it may stimulate others to evaluate the completeness and quality of notifications at local and regional level; such studies make it possible to develop the needed interventions to improve our surveillance systems.
The results of this study demonstrate that there is limited overlap in the cases notified to both of the systems. The sensitivity was calculated based on the total cases notified, with the assumption that no duplicate notification occurred within each of the systems, and that all cases of hepatitis A were indeed captured by at least one of the two systems. This is not a realistic assumption (for example, outpatients may be less likely to be notified than inpatients); for this reason the reported sensitivity may be overestimated. On the other hand, results may have been affected by the completeness of the record linkage, which may not have been optimal as a result of missing data or errors in data entry.
A factor that could have influenced the concordance in reporting was the fact that most of the cases were reported in 1997. The SEIEVA system was launched in Puglia in 1997; at the same time, a major hepatitis A epidemic was occurring in the region. The health care workers likely faced a heavy burden of reporting, and at the same time were just becoming familiar with the second surveillance system.
It would be useful to expand this project to other areas of the country to determine whether this is a local or more general problem. In addition, it would be useful to perform a more in-depth analysis to determine the extent of the undernotification, especially which procedures in the data collection or transmission that require review or modification. To determine the extent of undernotification, it might be feasible to conduct active surveillance, during a brief period, in a limited number of AUSLs, by contacting all the family doctors and hospitals or by adding a third information source such as the national hospital discharge records and examine the independence of these sources. This would allow assessment of cases notified to neither of the two systems.
To determine whether undernotification is a random event unlinked to the characteristics of the cases or to more systematic factors that may distort conclusions regarding the characteristics of hepatitis A patients, it would be helpful to evaluate the ways in which cases notified only to SIMI differ from those notified only to SEIEVA. For example, some AUSLs may notify to only one of the two systems or SIEIVA may include only hospitalised cases.
1) Mele A, Bianco E, Spada E, Ciccozzi M, Marzolini A, Tosti ME. SEIEVA Sistema Epidemiologico integrato dell’epatite virale acuta. Rapporto 1997-1998. Rapporti Istisan 2000; 20: 1-23.
2) Seber GAF. The effect of trap response on tag recapture estimates. Biometrics 1970; 26:13-22.
3) Germinario C, Lopalco PL, Chironna M, et al. From hepatitis B to hepatitis A and B prevention: the Puglia (Italy) experience. Vaccine 2000; 28 (1): 583-585
