On 2015 Feb 12, Tamás Ferenci commented:
EDIT (2015 Feb 13): I removed my comment for reasons of elegance, as Dr. Miller's original comment - to which I was solely replying - was also removed.
On 2015 Feb 12, Clifford Miller commented:
None
On 2015 Feb 11, Tamás Ferenci commented:
Dear Dr. Puliyel,
re: the TOKEN study. You quote Table 31, which (a) pertains not to the hexavalent vaccine, but rather hexa- and pentavalent combined (I'll return to the significance of this shortly), and (b) presents the results of the unweighted analysis.
Let us first examine what weighting means. It is possible to compensate for the preferential enrollment of those who were recently vaccinated (which is not only a theoretical possibility, rather, its existence has been empirically demonstrated in the study), by a method called inverse-probability weighting, thus providing better estimates. Not only I'm saying this, but the authors of the study themselves (p. 13):
The results obtained from these weighted analyses are regarded as more valid and are therefore presented in this summary. For reasons of completeness, the unweighted analyses are reported in the full study report in addition to the weighted risks estimated.
So, you were quoting the less valid results, which were only presented for the sake of completeness. Let us now go to your other source, Table 36. It is at least a weighted analysis, but the first problem still lingers: it is again not pertaining to hexavalent vaccine, but hexa- and pentavalent combined. The problem is, that the results are dominated by the effect of pentavalent vaccine (cf. Table 41), which is on the one hand not the issue discussed here, and on the other hand, it was also the more error-prone part of the study, given the fact that only 14 pentavalently vaccinated subjects were included (as compared to the almost hundred hexavalently vaccinated). For objectivity, I once again quote the authors of the study (p. 14):
In addition, response rate was especially high for parents whose deceased child was recently vaccinated with a pentavalent vaccine. This self-selection and the very low number of cases substantially limits interpretation of these findings.
Now let's move on to the issue that is actually discussed for this article, the hexavalent vaccine. Table 27 presents the results with the weighted analysis in the 1st and 2nd life-year combined: there is no increased risk neither with Model 1, nor with Model 2 (just to refer to your remark...), Table 28 shows the results for 1st year alone: there is no increased risk neither with Model 1, nor with Model 2. Tables 29 and 30 presents the results of the stratified analyses, none of which shows increased risk.
I also note that there was a case-control part in addition to the SCCS, which found no increased risk in any of the investigated scenarios.
re: reporting bias. I believe we have discussed everything here, just one very minor addition, which cropped up when I was browsing the comments. At another point, Dr. Miller is quite strongly arguing that 90% (!!!) of the adverse events, even serious adverse events, he specifically added (and used in the calculations for SIDS), don't get reported. Now 90% is not reported or these events are "investigated thoroughly" and thus "ascertainment bias is not likely to play a major role"...? You should decide..
On 2015 Feb 06, Jacob Puliyel commented:
None
On 2015 Feb 05, Tamás Ferenci commented:
Dear Dr. Puliyel,
Thanks for your response. Let me have a few comment on it.
If the reporting is so bad the clusters aren't real, then the data can't/shouldn't be used to defend its safety.
Or to question it... Actually, I very much do hope that it is not used, at least in itself, to "defend" its safety: like I already discussed, this data simply can't be used for such purposes, the sole application is to warrant the performing of active safety studies (which have been done).
3A data is self-reported. It is no crime if a parent does not report to the doctor that their child developed leg pain a few days after being administered Infanrix.
For the record, 3A includes only serious adverse events, ranging from ITP through haematochezia to anaphylaxis, and yes, it even includes deaths. While I agree that reports made by parents can not be compared to reports made by physicians, comparing such adverse events to "leg pain" is not too fair, either.
Forensic experts are unlikely to ‘forget’ to mention immunization, simply because it was not given on the day of death but on the previous day.
By this, we get back to the first half of the 1st remark in my original comment (which you have not addressed). Under the circumstances you described, we could very well argue that such "professional forensic experts" are also unlikely to forget a vaccination a week ago as well, yet, it can be easily demonstrated that in reality this needs to be the case, see my original comment.
But it is difficult to argue convincingly that higher under-reporting is likely on the day just after a vaccine is administered, compared to the day of vaccination.
Actually, we don't need to: Day 1 is not significantly different from Day 0, Day 2 is the same as Day 1, so the first point where the number of reports truly drops is at the third day.
Tables 31 and 36 show significantly increased risk of unexplained sudden unexpected death in the first 3 days after hexa- or pentavalent vaccination (1st and 2nd year of life).
While I am not specifically familiar with the TOKEN study, it is immediately clear that you only cited half of the results: those tables also reveal that while there is indeed a period of increased risk on day 0-3, but this is followed by a period (day 4-7) of decreased risk, the net effect of which is an unchanged risk...
So it appears that active studies have confirmed that there are two vaccines which cause 'sudden deaths'.
OK, so we have a range of studies cited by Dr. Franco (none of which you have addressed), all having negative result, a SINGLE study which you cited, whose results are summarized by its own authors as "[t]he main study analysis showed no increased risk of sudden death within one week after hexavalent vaccination [...] multivariate case-control analysis [...] was in accordance with this finding and did not suggest a risk increase within one week after hexavalent vaccination", and you described this situation as "active studies have confirmed that there are two vaccines which cause 'sudden deaths'". Well...
On 2015 Feb 04, Jacob Puliyel commented:
I thank Ferenci and Miller for their responses. I will address the three points made by Dr Ferenci
1) The data I have quoted (from Table 36) was made available by the manufacturers GSK, to defend the safety record of Infanrix Hexa to the regulatory authority (- the EMA). The data suggests a cluster on and following the day of vaccination. If the reporting is so bad the clusters aren't real, then the data can't/shouldn't be used to defend its safety. If the reporting is good, then the clusters are real too, and the vaccine looks unsafe.
2) Dr Ferenci writes “ ..it is immediately obvious from 3A appendices (pp. 301-522, pp. 857-1064) that no matter which disease-group we look at, the vast majority of spontaneous reports with known time are definitely coming from the first few days!”
The sources for the the reports in the 3A appendices are different. 3A data is self-reported. It is no crime if a parent does not report to the doctor that their child developed leg pain a few days after being administered Infanrix.
But SIDS deaths are different and have to be reported mandatorily to those like coroners who must determine the cause of death. They are investigated by professional forensic experts. SIDS are ‘deaths under suspicious circumstances’ - unexplained death that could be infanticide unless proved otherwise. Forensic experts are unlikely to ‘forget’ to mention immunization, simply because it was not given on the day of death but on the previous day. Reporting bias is less likely to be an issue with such forensic reports. Under-reporting on all days will of course still occur for all the reasons it occurs for other serious adverse events. But it is difficult to argue convincingly that higher under-reporting is likely on the day just after a vaccine is administered, compared to the day of vaccination.
3) Finally, Dr Ferenci says that active vaccine safety studies are better than passively acquired data. For well designed, managed and executed studies I wholeheartedly agree with him.
The TOKEN study aimed to assess comprehensively a possible causal relationship between vaccination and unexplained sudden unexpected death of children between their 2nd and 24th month of life. The study was supported and sponsored by the Paul-Ehrlich-Institute (PEI) and the Federal Ministry of Health (Bundesministerium für Gesundheit). Unfortunately this large study with a wealth of data has not been published in an indexed peer reviewed journal as yet. It is available here: http://www.rki.de/DE/Content/Gesundheitsmonitoring/Studien/Weitere_Studien/TOKEN_Studie/Studyreport.pdf?__blob=publicationFile
Parents of children who had died of SIDS were requested to participate in the study. 37.6% (254 cases) could be included in the study, where parental consent was obtained. Tables 31 and 36 show significantly increased risk of unexplained sudden unexpected death in the first 3 days after hexa- or pentavalent vaccination (1st and 2nd year of life).
So it appears that active studies have confirmed that there are two vaccines which cause 'sudden deaths'. I am grateful that the Italian Court has allowed public scrutiny of GSK's PSUR reports held as confidential by the EMA.
Jacob Puliyel
On 2015 Feb 08, Clifford Miller commented:
There is a matter to add to my above comments concerning the Observed/Expected analysis provided by GSK to the European Medicines Agency [EMA] and to my exchanges with Dr Ferenci here: http://www.ncbi.nlm.nih.gov/pubmed/24004825#cm24004825_8920
Where death and vaccination occurred on the same day, unlike any death occurring after the day of vaccination, only half the children who would have been expected to die by chance alone would have been vaccinated.
As vaccination appointments can take place at any time during the daytime, one has to adjust for the expected number of deaths by chance where the child died without being vaccinated.
Neither GSK nor the EMA appear to have considered that point judging by the information in GSK's PSUR's 15 and 16.
SIDS cases are often found in the early hours of the morning and can also die in their sleep at any time during daytime.
This would mean that half the number deaths expected by chance on Day 0 would have been of vaccinated children. So only half the number of spontaneous reports would be expected.
Spontaneous reports of adverse events cannot be relied upon to state the exact times of death nor of vaccination. The detailed narrative accounts of cases in GSK's PSURs 15 and 16 support this. So it seems unlikely that Day 0 means a period of 24 hours commencing from the time of vaccination but merely means the calendar day starting from and ending at midnight.
On such a basis, if all deaths occurred on the first dose the number expected by chance to die regardless of vaccination would be 162 and not GSK's 54. The data suggests one must weight the expected deaths as it seems more deaths were on the day of the first dose - namely on the same day as vaccination. So one would expect by chance alone a lower number than 162 deaths, but a higher number than 54.
Adjusting for underreporting running at 90% in a country, the UK, with better reporting rates than most other countries this brings us to expect no more than 16 spontaneous reports of sudden deaths to occur by chance on the day of first vaccination if all 162 deaths expected by chance were vaccinated.
But if only half the number of children who might be expected by chance to die are vaccinated [81] then one expects no more than 8 spontaneous reports of deaths associated with contemporaneous vaccination to be by chance compared to the 16 actual reports.
What is remarkable about that is 16 deaths were observed. This is in close correspondence within the same order of magnitude with a crude estimation of the number of spontaneous reports one might expect by chance to coincide with vaccination [ie. 8].
In addition, the healthy vaccinee effect being greatest on the first day of vaccination [1] indicates a low likelihood these are spurious mere coincidences. And as Dr Puliyel has observed, sudden deaths of children are investigated as potential cases of infanticide, unlike other kinds of deaths. And ~90% of the spontaneous reports were by medical professionals.
So one has a greater degree of confidence such reports are more reliable and that the reports are of children who died because they were vaccinated and not because it is mere coincidence. The numbers do not support the latter conclusion.
As noted previously, the proper comparison is not what GSK submitted to the EMA but between the number of spontaneous reports to be "Expected" by chance and those "Observed".
I note Dr Puliyel asked for peer review of his observations. I too ask the authors of this paper to do the same for those I have set out.
[1] https://www.ruor.uottawa.ca/bitstream/10393/31774/1/Hawken_Steven_2014_thesis.pdf
EDITED FOR GREATER CLARITY @ 14:30 GMT 8th Feb 2015
On 2015 Feb 05, Clifford Miller commented:
Whichever way one looks at this problem, the reports of sudden death do not look like they should be considered in any way coincidental.
Ignoring the observed clustering of reports on the day of and days following vaccination, if the GSK figure of 54 is not the number of deaths to be expected on any day but the number of deaths to be expected to coincide with any one of the three vaccinations in a 180 day vaccination period: 1) without correcting for underreporting; 2) assuming the rate of sudden death GSK used is applicable for the periods concerned; 3) the healthy vaccinee correction of 0.8 is applicable; and 4) this is not confounded by co-administration the same day with any other vaccine, then one would expect 54 deaths to coincide with the day of vaccination over the course of three vaccines.
It is not however the correct calculation for the number of deaths to be expected to be reported. And that is what GSK and the EMA should have compared.
Ignoring 2), 3) and 4), underreporting of serious adverse events to safety regulators is high. Correcting for underreporting using the rate observed in a country with good rates of reporting, then GSK's 54 figure becomes 5.4 expected reports [90% underreporting of serious adverse events] compared to 16 reports of observed actual deaths:
"Compared to other countries the number of spontaneous reports submitted in the UK is relatively high and reporting rates in relation to prescription volumes are also among the best in Europe.2 It is estimated, however, that only 10 per cent of serious reactions and between 2 and 4 per cent of non-serious reactions are reported.2 Such a high level of under-reporting will necessarily lead to bias in the data collected via the Yellow Card Scheme." BMA Board of Science Reporting adverse drug reactions A guide for healthcare professionals May 2006
Even if we assume a highly conservative figure for underreporting namely that 2/3 of serious adverse events are not reported, the GSK figure becomes 18 compared to the observed 16.
And the healthy vaccinee correction may be optimistic. A 2014 thesis published by the University of Ottawa shows that on the day of vaccination serious ill health in children is 1/3rd of that on any other day, so one would expect spurious reports of merely coincidental ill-health to be 1/3rd of those on any other day including being lower than on the day following vaccination:
https://www.ruor.uottawa.ca/bitstream/10393/31774/1/Hawken_Steven_2014_thesis.pdf
The position is confounded by Infanrix Hexa being co-administered with two other vaccines on the same day - so it is not the only vaccine to be considered. According to Wyeth's [now Pfizer] PSUR 4 for Prevenar 13 the rate of reported neurological complications is highest when co-administered with Infanrix Hexa, lowest when administered alone and when administered with any other vaccine the figure lies between the two. [1]
[1] Prevenar 13 : PSUR 04 – Response to RSI MA numbers: EU/1/09/590/001-006
On 2015 Feb 04, Clifford Miller commented:
My reply to Dr Ferenci [2015 Feb 03 2:20 p.m.] now appears beneath his comment [2015 Feb 02 09:30 a.m.]. The EMA's failure to notice GSK's error in calculating the number of sudden deaths per day predicted to be expected by chance as three times too high [54] instead of what it should be [18] is addressed. The population to a reasonable approximation for these purposes is not 54.7 million doses but 54.7m doses / 3 administered per child.
On 2015 Feb 03, Clifford Miller commented:
None
On 2015 Feb 06, Clifford Miller commented:
Dear Dr Ferenci,
Thank you. I am not repeating what you said. My comments are in the context of my overall comments to which I made express reference here:
http://www.ncbi.nlm.nih.gov/pubmed/24004825#cm24004825_8956
Perhaps you might like to summarise your responses to all the observations made in the context in which they were made. By such means we can see what is common ground.
On 2015 Feb 05, Tamás Ferenci commented:
Dear Dr. Clifford,
I believe you are practically repeating, almost literally, what I said in my very first reply. Let me quote myself:
[S]trictly speaking "all of which can result in an SIDS" is only true if these events are independent, which might not be the case, but assuming that such event can only and exclusively occur after the first vaccination (and the probability at the further vaccinations is zero, conditional on surviving the first), which would give rise to your calculation, is no less irrealistic in my opinion.
Thus, my conclusion is unchanged: Nevertheless, I agree with you that this issue would have worthed at least a discussion in the PSUR, it'd have been elegant to discuss it, but at this point, I don't see that it is "plain wrong", as suggested by your comment.
Let me also add that even if you are correct that most deaths happened after the first dose, this is still not a conclusive evidence per se: SIDS has a decreasing age-specific incidence curve at this age range, so higher dose repetition numbers coincide with higher age which is in turn associated with lower risk itself. I.e. these two are confounded.
On 2015 Feb 05, Clifford Miller commented:
Dear Dr Ferenci,
Thank you. If one assumes an equal probability of death per dose then exposure is more relevant. If death is more likely on the first dose then the population is more relevant.
See also my comment above: http://www.ncbi.nlm.nih.gov/pubmed/24004825#cm24004825_8956
GSK's 54 expected deaths figure is based on an equal likelihood on each exposure. This ignores the likelihood of death on the first dose. The data GSK provided to the EMA appears misleading. The figure for deaths expected by chance should be much much closer to 18 than to 54.
GSK's individual case report data in PSUR's 15 and 16 shows the following distribution of deaths for those reports which included the information:
7 @ 2 months, 8 @ 3 months, 1 @ 4 months, 2 @ 5 months, 1 @ 6 months 1 @ 11 months.
Some of the deaths aged 3 months may be first dose vaccinations given at 2 months or could also be late first doses at three months. The age at vaccination is not given.
The interval between vaccination and death for these deaths was between 0 and up to 14 days [with one at 24 days].
The EMA SPC dosage schedule for these ages states:
The primary vaccination schedule consists of three doses of 0.5 ml (such as 2, 3, 4 months; 3, 4, 5 months; 2, 4, 6 months) or two doses (such as 3, 5 months). There should be an interval of at least 1 month between doses.
The Expanded Program on Immunisation schedule (at 6, 10, 14 weeks of age) may only be used if a dose of hepatitis B vaccine has been given at birth.
On 2015 Feb 05, Tamás Ferenci commented:
Dear Dr. Miller,
cited in your comment 54 million as the population figure when the maximum upper limit is 18 million to a good approximation.
Like I have discussed in my comment, the question is not the size of population, but the exposure. These are different concepts; this was just the main point of my discussion.
On 2015 Feb 04, Clifford Miller commented:
Dr Ferenci,
Thank you for your response.
GSK and the EMA appear to have made the issue of an Observed/Expected analysis central to the matter. That appears the main point - what to a reasonable approximation is the Expected figure to compare to the Observed 16 real spontaneous reports of real deaths of real children occurring on Day 0 [the day of vaccination].
The correction required to the approach used by GSK is to use a realistic figure to a good approximation for the maximum population.
At the moment the position with that correction appears to be 18[E]:16[O] to a good approximation for these purposes [albeit without yet making a correction for underreporting].
I put the issue of the population to you because your comment states you are a senior lecturer in biostatistics and because of course you chose to comment on the matter and cited in your comment 54 million as the population figure when the maximum upper limit is 18 million to a good approximation.
As for your other points, in the light of the foregoing, in my view it is the GSK O/E analysis which needs to be addressed - all else at this time being surplusage.
It is of course open to any of the authors or others who have commented who also used the figure of 54 million as representing the size of the population to comment.
On 2015 Feb 04, Tamás Ferenci commented:
Dear Dr. Miller,
Thanks for your remark.
Let me note, however (as you submitted your comment as a reply to mine), that my 2nd comment and the second half of my 1st comment is in no way dependent on the size of the background population, so they remain completely intact, even if your criticism is correct. The first half of my first comment indeed depends on the background size, but - as I have pointed out - the discrepancy is in excess to two magnitudes, so a factor of 3, even if you are correct, doesn't affect qualitatively my conclusion.
As a side note, I also can't completely agree with your logic. You are correct that this way, the same child is counted several times in the background population, but don't forget that the same child - save for those who die, but their number is of course negligible even compared to 18 million - is also exposed several times! (All of which can result in an SIDS and those results are counted together in Table 36.) Well, strictly speaking "all of which can result in an SIDS" is only true if these events are independent, which might not be the case, but assuming that such event can only and exclusively occur after the first vaccination (and the probability at the further vaccinations is zero, conditional on surviving the first), which would give rise to your calculation, is no less irrealistic in my opinion.
Nevertheless, I agree with you that this issue would have worthed at least a discussion in the PSUR, it'd have been elegant to discuss it, but at this point, I don't see that it is "plain wrong", as suggested by your comment.
On 2015 Feb 03, Clifford Miller commented:
Dr Ferenci,
The number of daily sudden deaths to be expected by chance is just in excess of 18 and not the number GSK reported to the EMA. One must not equate the number of doses sold [54.7 million] with the size of the population. The maximum vaccinated population is 18 million.
This means the number of sudden deaths predicted by chance using GSK's proposed annual rate of .454 per 1000 and healthy vaccinee factor of 0.8 is 18.
To that you must compare the number reported in spontaneous reports. That number according to GSK is 16 dead children, [and that is without correcting for underreporting].
Of the number of doses sold GSK estimate the number used for Year 1 is 54.7 million. Each child historically was to receive three doses according to the EMA Schedule. The authors of this paper suggest a 4 dose schedule is being adopted so 18 million is an upper boundary and that ignores over-vaccination, spoils and other wastage.
It is therefore a little troubling that the report to the regulator was so far in error and that the EMA appears to have failed to have noticed.
On 2015 Feb 02, Tamás Ferenci commented:
Dr. Puliyel’s comment raises important questions about the time clustering of adverse events observed in passive vaccine safety data, i.e. the pattern of temporal association of adverse events with vaccination – which is the only thing that can be investigated using such data –, however his approach neglects an important aspect thus leading to unfounded conclusions in my opinion.
The major issue was raised by Dr. Franco, but let me add two further remarks.
1) While it is obviously clear for everyone that we will never be able to surely tell the exact role of reporting bias (and thus the real effect of the vaccine on SIDS, if there is any), there are two points that worth mentioning in my opinion, which were not raised by Dr. Franco:
2) Finally, let me note that I have a feeling that Dr. Puliyel misunderstands the applicability of passive data. Due to the inherent bias outlined above, such data simply can not be used alone to make conclusions on causality. Rather, their aim – among others – is to detect “suspicious” cases, which can be then specifically investigated with appropriately designed active vaccine safety methods, which are capable – as opposed to what we can do from passive data – to draw causal conclusions. There is no point in claiming that by applying “better” methods, we detected causality from passive data, because this is simply impossible, the only thing that can be meaningfully asked for if we detect something suspicious (using any method, including Dr. Puliyel’s) is to perform an active study. But these have been conducted (as pointed out by Dr. Franco)! Now Dr. Puliyel either accepts their results, or questions their methodology (which is a separate issue), the only thing he can’t do is to question the results of an active study based on passive data.
Conflict of interest: None.
Tamás Ferenci PhD
Senior lecturer (biostatistics), Óbuda University, Budapest, Hungary
John von Neumann Faculty of Informatics, Physiological Controls Group
On 2015 Jan 22, Jacob Puliyel commented:
I thank the authors for their response to my comment.
I was presuming that in the countries from where this data was gathered, sudden infant death or SIDS is considered 'unnatural death'. If that is so, these deaths will have been investigated by a competent forensic team and the immunization records will have been examined to check if the infant was up to date with its vaccinations or whether there was an element of neglect. Reporting bias (based on parents perception that vaccine was the trigger for events that lead to the death of their child) would have little or no role under these circumstances.
The authors quote from the 'Guidelines for good pharmacovigilance practices'; that events that are expected, common and mild, or occur late after vaccination, are less likely to be reported. That is not applicable here, as SIDS is completely unexpected and a catastrophic event.
Further, the analysis in Table 2 of the linked article http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version) shows that there were 42 deaths in the first 3 days and only 16 in the next 3 days. It is difficult to imagine that reporting bias is responsible for this big a change, in so short a time.
We will need to find a more plausible explanation. Otherwise we have to accept that the deaths were caused by the vaccine and the diagnosis of SIDS was wrong.
Jacob Puliyel
On 2015 Jan 21, Elisabetta Franco commented:
Dr Puliyel’s analysis uses the period of 10-19 days post-vaccination as a control for the period of 0-9 days post-vaccination to determine whether there is an excess of Sudden Infant Deaths (SIDS) in the first ten days following administration of Infanrix-hexa. The opinion of all the Authors is that the suggested imbalance in reported SIDS between 0-9 days and 10-19 days periods represents a well recognised bias in spontaneous report reporting, where the shorter the time that has elapsed between the vaccination procedure and the event, the more likely it is to be perceived as a trigger and subsequently be reported. Conversely, events that are expected, common and mild, or occur late after vaccination, are less likely to be reported [1]. Many studies have demonstrated an absence of causal association between vaccination and SIDS [2,3]. Post marketing data are shared on a regular basis with regulatory authorities worldwide [1,4], who support this position [2,3].
On 2015 Jan 19, Jacob Puliyel commented:
Apropos the earlier posting there are a couple of other facts that we must consider when looking at the incidence of sudden unexplained deaths immediately following vaccination with Infanrix.
a) The safety assessment document has used the number of doses of vaccine distributed as the denominator. The report acknowledges that all the doses of the vaccine distributed, need not have been utilized.
There can be another argument against using this denominator. As each child is given up to 5 doses (https://www.gsksource.com/gskprm/htdocs/documents/INFANRIX.PDF) and they could die after any one of the doses (and you can die only once), perhaps it would be more appropriate to look at the number of deaths against the number of babies vaccinated (rather than the number of units of vaccine distributed). The appropriate denominator would be about one fifth the denominator used in the report.
b) Appendix 5A in the document sent to the regulator gives the International Event Report in 13 fatal cases. It can be seen in this sample that there were more deaths after the first dose than after the second and more after the second than after the third dose. This is a pattern seen with adverse events following immunization (AEFI) that are causatively related.
c) In May 2005, Zinka and colleagues have reported six cases of sudden infant deaths caused by another hexavalent vaccine (similar to Infanrix), called Hexavac Zinka B, 2006. Marketing authorization in the European Union was withdrawn in August 2005 (Doc.Ref.EMEA/207369/2005).
d) The CIOMS /WHO have revised the widely used Brighton Protocol for assessment of AEFI. The new scheme facilitates misclassification of vaccine related deaths as [Not an AEFI] and this has been discussed on PubMed Commons earlier. (http://www.ncbi.nlm.nih.gov/pubmed/19061929 ) (http://www.ncbi.nlm.nih.gov/pubmed/23452584 ) (http://www.ncbi.nlm.nih.gov/pubmed/24021304 ).
e) In some ways the deaths with Infanrix is similar to deaths seen with the use in Asia of Pentavalent vaccine against 5 disease ( DPT, hepatitis B, Hib) Puliyel J, 2013. Some of these deaths have been investigated by the WHO using this revised method and the vaccine had been declared safe. http://www.who.int/vaccine_safety/committee/topics/hpv/GACVSstatement_pentavalent_June2013.pdf
f) The deaths are completely unnecessary as the vaccines could have been given separately, and separately they have a long track record of safety. One hopes that the findings will result in an honest assessment of the harms being done by these new combined vaccines.
Conclusion
As mentioned earlier there is nothing sacrosanct about the original Brighton Classification (http://www.who.int/vaccine_safety/publications/AEFI_aide_memoire.pdf) but one has to evaluate the two schemes (Brighton vs CIOMS) from the point of view of patient safety to see which scheme would react to rare vaccine related adverse reaction signals early. “The causality scheme that insists on calling all reactions as ‘indeterminate’ or ‘inconsistent/coincidental’ just because they were not noticed in the original small clinical trials, undermines the very raison d'être of post marketing surveillance. Patient safety (meaning protecting patients) rather than vaccine safety (protecting vaccines) should be more important.”
On 2015 Jan 13, Jacob Puliyel commented:
Baldo and colleagues quote 2 references to suggest that in Germany, a population-based evaluation demonstrated a possible safety signal for DTPa-HBV-IPV-Hib-SP but failed to show an imbalance between observed and expected SUD cases for DTPa-HBV-IPV/Hib von Kries R, 2005 von Kries R, 2006. However this seems to be contradicted by the data that was submitted by the manufacturer to the regulatory authority and the analysis below.
The GlaxoSmithKline Biological Clinical Safety and Pharmacovigilance’s confidential report to the Regulatory Authority on Infanrix hexa (combined Diptheria Tetanus and Acelluar Pertusis, Hepatitis B, inactivated Poliomyelitis and Haemophilus influenza type B vaccine for the period 23 October 2009 to 22 October 2011 (the 15th and 16th Periodic Safety Update Report (PSUR)) has been made available to the public by the Italian Court of Justice Nicola Di Leo and is now available on the internet (http://autismoevaccini.files.wordpress.com/2012/12/vaccin-dc3a9cc3a8s.pdf)
Section 9.3.1.1 on pages 246-249 documents an evaluation of whether the number of ‘sudden deaths’ reported, exceeded the number one could expect to occur by coincidence - that is from the natural background incidence of sudden death. The background incidence of 0.454/1000 live births in the first year and 0.062/1000 live births is used, with a healthy vaccine correlation factor of 0.8 applied. Table 36 on page 249 tabulates the number of sudden death that would be expected to occur by chance within a range of days post vaccination.
Table 1 Cumulative number of observed and expected cases of Sudden Death following Infanrix hexa in children in their first or second year of life
This is available here: http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version)
( Source: Table 36 The GlaxoSmithKline Biological Clinical Safety and Pharmacovigilance report to Regulatory Authority )
According to this analysis, the number of sudden death cases reported after vaccination with Infantrix hexa is below the number of cases expected in children in the first year of life. It is equal or below the number of cases expected in children in the 2nd year of life.
However if one analyses the data looking at deaths in first 10 days after administration of vaccine and compares it to the deaths in the next 10 days, it is clear that 97% of deaths (65 deaths) in the infants below 1 year, occur in the first 10 days and 3% (2 deaths) occur in the next 10 days. Had the deaths been coincidental SIDS deaths unrelated to vaccination, the numbers of deaths in the two 10 day periods should have been the same.
Similarly in children older than 1 year, 87.5% deaths (7 deaths) occurred in the first 10 days and 12.5% (1 death) occurred in the next 10 days.
If we consider the number of deaths in the second 10-day-window-period as the baseline SIDS rate in these healthy children coming for immunization, we can see that there was an excess of 63 (65 – 2 = 63) deaths in the first year and excess of 6 deaths (7 – 1 = 6) among those vaccinated between 1 and 2 years.
In the reporting period, one must conclude that Infanrix hexa vaccine could have been responsible for at least 69 deaths.These are all deaths within a small window period (of 3 weeks) after a catastrophic event which has been investigated thoroughly (forensic investigation of sudden unexpected deaths - SIDS/SUDS), therefor ascertainment bias is unlikely to have played a major role.
Table 2 The daily increment in Sudden Death following Infanrix hexa in children ' is tabulated and made available here: http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version)
The decelerating incremental-deaths further supports the contention that there is a clear relationship of ‘sudden death’ to the vaccination episode. 42 deaths had taken place in the first three days after vaccination, 16 deaths in the next 3 days between day 3 and day 5, 3 deaths between day 6 and day 8, 2 deaths between day 9 and day 11, and there were only 2 deaths in all of the remaining 10 days. The fact that rate of deaths decreases rapidly and continuously as time elapses after immunization, makes it clear that the deaths are related to the vaccination episode.
This is being posted on PubMed Commons to put it up for open review by the scientific community, on account of its urgency, as this is a matter that involves the lives of children and there is a continuing risk to children.
As the authors of this article are best qualified to peer review this submission, I am inviting each of the authors castrom@wustl.edu paolo.bonanni@unifi.it mclaudia@fei.edu.br giovanni.gabutti@unife.it franco@med.uniroma2.it fem75838@gsk.com r.prato@unifg.it fvitale@igiene.unipa.it to review it and to post their review on PubMed Commons.
Jacob Puliyel MD MRCP M Phil
On 2015 Jan 13, Jacob Puliyel commented:
Baldo and colleagues quote 2 references to suggest that in Germany, a population-based evaluation demonstrated a possible safety signal for DTPa-HBV-IPV-Hib-SP but failed to show an imbalance between observed and expected SUD cases for DTPa-HBV-IPV/Hib von Kries R, 2005 von Kries R, 2006. However this seems to be contradicted by the data that was submitted by the manufacturer to the regulatory authority and the analysis below.
The GlaxoSmithKline Biological Clinical Safety and Pharmacovigilance’s confidential report to the Regulatory Authority on Infanrix hexa (combined Diptheria Tetanus and Acelluar Pertusis, Hepatitis B, inactivated Poliomyelitis and Haemophilus influenza type B vaccine for the period 23 October 2009 to 22 October 2011 (the 15th and 16th Periodic Safety Update Report (PSUR)) has been made available to the public by the Italian Court of Justice Nicola Di Leo and is now available on the internet (http://autismoevaccini.files.wordpress.com/2012/12/vaccin-dc3a9cc3a8s.pdf)
Section 9.3.1.1 on pages 246-249 documents an evaluation of whether the number of ‘sudden deaths’ reported, exceeded the number one could expect to occur by coincidence - that is from the natural background incidence of sudden death. The background incidence of 0.454/1000 live births in the first year and 0.062/1000 live births is used, with a healthy vaccine correlation factor of 0.8 applied. Table 36 on page 249 tabulates the number of sudden death that would be expected to occur by chance within a range of days post vaccination.
Table 1 Cumulative number of observed and expected cases of Sudden Death following Infanrix hexa in children in their first or second year of life
This is available here: http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version)
( Source: Table 36 The GlaxoSmithKline Biological Clinical Safety and Pharmacovigilance report to Regulatory Authority )
According to this analysis, the number of sudden death cases reported after vaccination with Infantrix hexa is below the number of cases expected in children in the first year of life. It is equal or below the number of cases expected in children in the 2nd year of life.
However if one analyses the data looking at deaths in first 10 days after administration of vaccine and compares it to the deaths in the next 10 days, it is clear that 97% of deaths (65 deaths) in the infants below 1 year, occur in the first 10 days and 3% (2 deaths) occur in the next 10 days. Had the deaths been coincidental SIDS deaths unrelated to vaccination, the numbers of deaths in the two 10 day periods should have been the same.
Similarly in children older than 1 year, 87.5% deaths (7 deaths) occurred in the first 10 days and 12.5% (1 death) occurred in the next 10 days.
If we consider the number of deaths in the second 10-day-window-period as the baseline SIDS rate in these healthy children coming for immunization, we can see that there was an excess of 63 (65 – 2 = 63) deaths in the first year and excess of 6 deaths (7 – 1 = 6) among those vaccinated between 1 and 2 years.
In the reporting period, one must conclude that Infanrix hexa vaccine could have been responsible for at least 69 deaths.These are all deaths within a small window period (of 3 weeks) after a catastrophic event which has been investigated thoroughly (forensic investigation of sudden unexpected deaths - SIDS/SUDS), therefor ascertainment bias is unlikely to have played a major role.
Table 2 The daily increment in Sudden Death following Infanrix hexa in children ' is tabulated and made available here: http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version)
The decelerating incremental-deaths further supports the contention that there is a clear relationship of ‘sudden death’ to the vaccination episode. 42 deaths had taken place in the first three days after vaccination, 16 deaths in the next 3 days between day 3 and day 5, 3 deaths between day 6 and day 8, 2 deaths between day 9 and day 11, and there were only 2 deaths in all of the remaining 10 days. The fact that rate of deaths decreases rapidly and continuously as time elapses after immunization, makes it clear that the deaths are related to the vaccination episode.
This is being posted on PubMed Commons to put it up for open review by the scientific community, on account of its urgency, as this is a matter that involves the lives of children and there is a continuing risk to children.
As the authors of this article are best qualified to peer review this submission, I am inviting each of the authors castrom@wustl.edu paolo.bonanni@unifi.it mclaudia@fei.edu.br giovanni.gabutti@unife.it franco@med.uniroma2.it fem75838@gsk.com r.prato@unifg.it fvitale@igiene.unipa.it to review it and to post their review on PubMed Commons.
Jacob Puliyel MD MRCP M Phil
On 2015 Jan 19, Jacob Puliyel commented:
Apropos the earlier posting there are a couple of other facts that we must consider when looking at the incidence of sudden unexplained deaths immediately following vaccination with Infanrix.
a) The safety assessment document has used the number of doses of vaccine distributed as the denominator. The report acknowledges that all the doses of the vaccine distributed, need not have been utilized.
There can be another argument against using this denominator. As each child is given up to 5 doses (https://www.gsksource.com/gskprm/htdocs/documents/INFANRIX.PDF) and they could die after any one of the doses (and you can die only once), perhaps it would be more appropriate to look at the number of deaths against the number of babies vaccinated (rather than the number of units of vaccine distributed). The appropriate denominator would be about one fifth the denominator used in the report.
b) Appendix 5A in the document sent to the regulator gives the International Event Report in 13 fatal cases. It can be seen in this sample that there were more deaths after the first dose than after the second and more after the second than after the third dose. This is a pattern seen with adverse events following immunization (AEFI) that are causatively related.
c) In May 2005, Zinka and colleagues have reported six cases of sudden infant deaths caused by another hexavalent vaccine (similar to Infanrix), called Hexavac Zinka B, 2006. Marketing authorization in the European Union was withdrawn in August 2005 (Doc.Ref.EMEA/207369/2005).
d) The CIOMS /WHO have revised the widely used Brighton Protocol for assessment of AEFI. The new scheme facilitates misclassification of vaccine related deaths as [Not an AEFI] and this has been discussed on PubMed Commons earlier. (http://www.ncbi.nlm.nih.gov/pubmed/19061929 ) (http://www.ncbi.nlm.nih.gov/pubmed/23452584 ) (http://www.ncbi.nlm.nih.gov/pubmed/24021304 ).
e) In some ways the deaths with Infanrix is similar to deaths seen with the use in Asia of Pentavalent vaccine against 5 disease ( DPT, hepatitis B, Hib) Puliyel J, 2013. Some of these deaths have been investigated by the WHO using this revised method and the vaccine had been declared safe. http://www.who.int/vaccine_safety/committee/topics/hpv/GACVSstatement_pentavalent_June2013.pdf
f) The deaths are completely unnecessary as the vaccines could have been given separately, and separately they have a long track record of safety. One hopes that the findings will result in an honest assessment of the harms being done by these new combined vaccines.
Conclusion
As mentioned earlier there is nothing sacrosanct about the original Brighton Classification (http://www.who.int/vaccine_safety/publications/AEFI_aide_memoire.pdf) but one has to evaluate the two schemes (Brighton vs CIOMS) from the point of view of patient safety to see which scheme would react to rare vaccine related adverse reaction signals early. “The causality scheme that insists on calling all reactions as ‘indeterminate’ or ‘inconsistent/coincidental’ just because they were not noticed in the original small clinical trials, undermines the very raison d'être of post marketing surveillance. Patient safety (meaning protecting patients) rather than vaccine safety (protecting vaccines) should be more important.”
On 2015 Jan 21, Elisabetta Franco commented:
Dr Puliyel’s analysis uses the period of 10-19 days post-vaccination as a control for the period of 0-9 days post-vaccination to determine whether there is an excess of Sudden Infant Deaths (SIDS) in the first ten days following administration of Infanrix-hexa. The opinion of all the Authors is that the suggested imbalance in reported SIDS between 0-9 days and 10-19 days periods represents a well recognised bias in spontaneous report reporting, where the shorter the time that has elapsed between the vaccination procedure and the event, the more likely it is to be perceived as a trigger and subsequently be reported. Conversely, events that are expected, common and mild, or occur late after vaccination, are less likely to be reported [1]. Many studies have demonstrated an absence of causal association between vaccination and SIDS [2,3]. Post marketing data are shared on a regular basis with regulatory authorities worldwide [1,4], who support this position [2,3].
On 2015 Jan 22, Jacob Puliyel commented:
I thank the authors for their response to my comment.
I was presuming that in the countries from where this data was gathered, sudden infant death or SIDS is considered 'unnatural death'. If that is so, these deaths will have been investigated by a competent forensic team and the immunization records will have been examined to check if the infant was up to date with its vaccinations or whether there was an element of neglect. Reporting bias (based on parents perception that vaccine was the trigger for events that lead to the death of their child) would have little or no role under these circumstances.
The authors quote from the 'Guidelines for good pharmacovigilance practices'; that events that are expected, common and mild, or occur late after vaccination, are less likely to be reported. That is not applicable here, as SIDS is completely unexpected and a catastrophic event.
Further, the analysis in Table 2 of the linked article http://jacob.puliyel.com/paper.php?id=345 (Please download pdf version) shows that there were 42 deaths in the first 3 days and only 16 in the next 3 days. It is difficult to imagine that reporting bias is responsible for this big a change, in so short a time.
We will need to find a more plausible explanation. Otherwise we have to accept that the deaths were caused by the vaccine and the diagnosis of SIDS was wrong.
Jacob Puliyel
On 2015 Feb 02, Tamás Ferenci commented:
Dr. Puliyel’s comment raises important questions about the time clustering of adverse events observed in passive vaccine safety data, i.e. the pattern of temporal association of adverse events with vaccination – which is the only thing that can be investigated using such data –, however his approach neglects an important aspect thus leading to unfounded conclusions in my opinion.
The major issue was raised by Dr. Franco, but let me add two further remarks.
1) While it is obviously clear for everyone that we will never be able to surely tell the exact role of reporting bias (and thus the real effect of the vaccine on SIDS, if there is any), there are two points that worth mentioning in my opinion, which were not raised by Dr. Franco:
2) Finally, let me note that I have a feeling that Dr. Puliyel misunderstands the applicability of passive data. Due to the inherent bias outlined above, such data simply can not be used alone to make conclusions on causality. Rather, their aim – among others – is to detect “suspicious” cases, which can be then specifically investigated with appropriately designed active vaccine safety methods, which are capable – as opposed to what we can do from passive data – to draw causal conclusions. There is no point in claiming that by applying “better” methods, we detected causality from passive data, because this is simply impossible, the only thing that can be meaningfully asked for if we detect something suspicious (using any method, including Dr. Puliyel’s) is to perform an active study. But these have been conducted (as pointed out by Dr. Franco)! Now Dr. Puliyel either accepts their results, or questions their methodology (which is a separate issue), the only thing he can’t do is to question the results of an active study based on passive data.
Conflict of interest: None.
Tamás Ferenci PhD
Senior lecturer (biostatistics), Óbuda University, Budapest, Hungary
John von Neumann Faculty of Informatics, Physiological Controls Group
On 2015 Feb 03, Clifford Miller commented:
None
On 2015 Feb 04, Clifford Miller commented:
My reply to Dr Ferenci [2015 Feb 03 2:20 p.m.] now appears beneath his comment [2015 Feb 02 09:30 a.m.]. The EMA's failure to notice GSK's error in calculating the number of sudden deaths per day predicted to be expected by chance as three times too high [54] instead of what it should be [18] is addressed. The population to a reasonable approximation for these purposes is not 54.7 million doses but 54.7m doses / 3 administered per child.
On 2015 Feb 04, Jacob Puliyel commented:
I thank Ferenci and Miller for their responses. I will address the three points made by Dr Ferenci
1) The data I have quoted (from Table 36) was made available by the manufacturers GSK, to defend the safety record of Infanrix Hexa to the regulatory authority (- the EMA). The data suggests a cluster on and following the day of vaccination. If the reporting is so bad the clusters aren't real, then the data can't/shouldn't be used to defend its safety. If the reporting is good, then the clusters are real too, and the vaccine looks unsafe.
2) Dr Ferenci writes “ ..it is immediately obvious from 3A appendices (pp. 301-522, pp. 857-1064) that no matter which disease-group we look at, the vast majority of spontaneous reports with known time are definitely coming from the first few days!”
The sources for the the reports in the 3A appendices are different. 3A data is self-reported. It is no crime if a parent does not report to the doctor that their child developed leg pain a few days after being administered Infanrix.
But SIDS deaths are different and have to be reported mandatorily to those like coroners who must determine the cause of death. They are investigated by professional forensic experts. SIDS are ‘deaths under suspicious circumstances’ - unexplained death that could be infanticide unless proved otherwise. Forensic experts are unlikely to ‘forget’ to mention immunization, simply because it was not given on the day of death but on the previous day. Reporting bias is less likely to be an issue with such forensic reports. Under-reporting on all days will of course still occur for all the reasons it occurs for other serious adverse events. But it is difficult to argue convincingly that higher under-reporting is likely on the day just after a vaccine is administered, compared to the day of vaccination.
3) Finally, Dr Ferenci says that active vaccine safety studies are better than passively acquired data. For well designed, managed and executed studies I wholeheartedly agree with him.
The TOKEN study aimed to assess comprehensively a possible causal relationship between vaccination and unexplained sudden unexpected death of children between their 2nd and 24th month of life. The study was supported and sponsored by the Paul-Ehrlich-Institute (PEI) and the Federal Ministry of Health (Bundesministerium für Gesundheit). Unfortunately this large study with a wealth of data has not been published in an indexed peer reviewed journal as yet. It is available here: http://www.rki.de/DE/Content/Gesundheitsmonitoring/Studien/Weitere_Studien/TOKEN_Studie/Studyreport.pdf?__blob=publicationFile
Parents of children who had died of SIDS were requested to participate in the study. 37.6% (254 cases) could be included in the study, where parental consent was obtained. Tables 31 and 36 show significantly increased risk of unexplained sudden unexpected death in the first 3 days after hexa- or pentavalent vaccination (1st and 2nd year of life).
So it appears that active studies have confirmed that there are two vaccines which cause 'sudden deaths'. I am grateful that the Italian Court has allowed public scrutiny of GSK's PSUR reports held as confidential by the EMA.
Jacob Puliyel
On 2015 Feb 05, Tamás Ferenci commented:
Dear Dr. Puliyel,
Thanks for your response. Let me have a few comment on it.
If the reporting is so bad the clusters aren't real, then the data can't/shouldn't be used to defend its safety.
Or to question it... Actually, I very much do hope that it is not used, at least in itself, to "defend" its safety: like I already discussed, this data simply can't be used for such purposes, the sole application is to warrant the performing of active safety studies (which have been done).
3A data is self-reported. It is no crime if a parent does not report to the doctor that their child developed leg pain a few days after being administered Infanrix.
For the record, 3A includes only serious adverse events, ranging from ITP through haematochezia to anaphylaxis, and yes, it even includes deaths. While I agree that reports made by parents can not be compared to reports made by physicians, comparing such adverse events to "leg pain" is not too fair, either.
Forensic experts are unlikely to ‘forget’ to mention immunization, simply because it was not given on the day of death but on the previous day.
By this, we get back to the first half of the 1st remark in my original comment (which you have not addressed). Under the circumstances you described, we could very well argue that such "professional forensic experts" are also unlikely to forget a vaccination a week ago as well, yet, it can be easily demonstrated that in reality this needs to be the case, see my original comment.
But it is difficult to argue convincingly that higher under-reporting is likely on the day just after a vaccine is administered, compared to the day of vaccination.
Actually, we don't need to: Day 1 is not significantly different from Day 0, Day 2 is the same as Day 1, so the first point where the number of reports truly drops is at the third day.
Tables 31 and 36 show significantly increased risk of unexplained sudden unexpected death in the first 3 days after hexa- or pentavalent vaccination (1st and 2nd year of life).
While I am not specifically familiar with the TOKEN study, it is immediately clear that you only cited half of the results: those tables also reveal that while there is indeed a period of increased risk on day 0-3, but this is followed by a period (day 4-7) of decreased risk, the net effect of which is an unchanged risk...
So it appears that active studies have confirmed that there are two vaccines which cause 'sudden deaths'.
OK, so we have a range of studies cited by Dr. Franco (none of which you have addressed), all having negative result, a SINGLE study which you cited, whose results are summarized by its own authors as "[t]he main study analysis showed no increased risk of sudden death within one week after hexavalent vaccination [...] multivariate case-control analysis [...] was in accordance with this finding and did not suggest a risk increase within one week after hexavalent vaccination", and you described this situation as "active studies have confirmed that there are two vaccines which cause 'sudden deaths'". Well...
On 2015 Feb 06, Jacob Puliyel commented:
None
On 2015 Feb 11, Tamás Ferenci commented:
Dear Dr. Puliyel,
re: the TOKEN study. You quote Table 31, which (a) pertains not to the hexavalent vaccine, but rather hexa- and pentavalent combined (I'll return to the significance of this shortly), and (b) presents the results of the unweighted analysis.
Let us first examine what weighting means. It is possible to compensate for the preferential enrollment of those who were recently vaccinated (which is not only a theoretical possibility, rather, its existence has been empirically demonstrated in the study), by a method called inverse-probability weighting, thus providing better estimates. Not only I'm saying this, but the authors of the study themselves (p. 13):
The results obtained from these weighted analyses are regarded as more valid and are therefore presented in this summary. For reasons of completeness, the unweighted analyses are reported in the full study report in addition to the weighted risks estimated.
So, you were quoting the less valid results, which were only presented for the sake of completeness. Let us now go to your other source, Table 36. It is at least a weighted analysis, but the first problem still lingers: it is again not pertaining to hexavalent vaccine, but hexa- and pentavalent combined. The problem is, that the results are dominated by the effect of pentavalent vaccine (cf. Table 41), which is on the one hand not the issue discussed here, and on the other hand, it was also the more error-prone part of the study, given the fact that only 14 pentavalently vaccinated subjects were included (as compared to the almost hundred hexavalently vaccinated). For objectivity, I once again quote the authors of the study (p. 14):
In addition, response rate was especially high for parents whose deceased child was recently vaccinated with a pentavalent vaccine. This self-selection and the very low number of cases substantially limits interpretation of these findings.
Now let's move on to the issue that is actually discussed for this article, the hexavalent vaccine. Table 27 presents the results with the weighted analysis in the 1st and 2nd life-year combined: there is no increased risk neither with Model 1, nor with Model 2 (just to refer to your remark...), Table 28 shows the results for 1st year alone: there is no increased risk neither with Model 1, nor with Model 2. Tables 29 and 30 presents the results of the stratified analyses, none of which shows increased risk.
I also note that there was a case-control part in addition to the SCCS, which found no increased risk in any of the investigated scenarios.
re: reporting bias. I believe we have discussed everything here, just one very minor addition, which cropped up when I was browsing the comments. At another point, Dr. Miller is quite strongly arguing that 90% (!!!) of the adverse events, even serious adverse events, he specifically added (and used in the calculations for SIDS), don't get reported. Now 90% is not reported or these events are "investigated thoroughly" and thus "ascertainment bias is not likely to play a major role"...? You should decide..
On 2015 Feb 12, Clifford Miller commented:
None
On 2015 Feb 12, Tamás Ferenci commented:
EDIT (2015 Feb 13): I removed my comment for reasons of elegance, as Dr. Miller's original comment - to which I was solely replying - was also removed.