On 2017 Jun 12, Daniel Weeks commented:

As Neuron appears to have deleted our original comment, here is a copy for the record.

NR1H3 and multiple sclerosis: questionable assumptions and miscalculated p-values

Wang et al (2016) investigated the role of the nuclear receptor NR1H3 in familial multiple sclerosis (MS), and described two of the highlights of their research as (1) “An arginine to glutamine mutation in NR1H3 causes multiple sclerosis in families” and (2) “Common variants in NR1H3 are associated with primary progressive multiple sclerosis”. Regarding the first claim, in a comment on PubMed Commons (http://www.ncbi.nlm.nih.gov/pubmed/27253448#cm27253448_16159), Eric Vallabh Minikel and Daniel MacArthur raised concerns on the basis of the frequencies of the implicated NR1H3 variant, rs61731956, encoding p.Arg415Gln, in the Exome Aggregation Consortium (ExAC). Minikel and MacArthur point out that “the variant is not significantly enriched in cases over ExAC population controls (P = .56) - indeed, its allele frequency is lower in MS cases (0.02%) than in ExAC European population controls (0.03%)”.

As evidence of co-segregation of rs61731956 with disease, Wang et al (2016) report a maximum LOD score of 2.20 at θ=0. However, this LOD score was computed under a fully penetrant model where “unaffected mutation carriers were treated as having an unknown disease status”. This overstates the evidence for co-segregation because in linkage analysis disease status should be assigned blind to mutation status, and so a reduced penetrance model should have instead been used where unaffected individuals were properly coded as having an unaffected disease status. Indeed, later in their manuscript, Wang et al (2016) cite the presence of “three obligate carriers and an unaffected biological family member” as evidence of incomplete penetrance.

The evidence for the second claim, that “common variants in NR1H3 are associated with primary progressive multiple sclerosis”, is also overstated in Wang et al (2016) because the p-values in their Table 1 were incorrectly computed. The p-values presented were derived by computing a 2 degree of freedom chi-squared statistic based on the 3 x 2 genotype table and then looking up the p-value of that statistic using a 1 degree of freedom distribution. In our Table 1, we present the correct p-values for the 2 degree of freedom chi-squared statistic. But as some of the cell counts are small, it would be more appropriate to use a Fisher’s Exact Test. Using the p-values of a Fisher’s Exact Test, and applying a Bonferroni’s correction for the 15 tests carried out (instead of correcting for only 5 as Wang et al did), none of the findings are significant at the 0.05 level after correction for multiple testing.

Based on the concerns raised by Minikel and MacArthur as well as here, it seems that the original claims were over-stated, and it is likely, based on the data presented, that this variant and gene may play no significant roles in MS. Of course, the collection of additional independent data followed by careful and correct statistical analyses will ultimately clarify whether or not NR1H3 plays a role in MS risk. Indeed the International MS Genetics Consortium has already examined this variant in their data (http://biorxiv.org/content/early/2016/07/01/061366), and found “no evidence that this variant is associated either with MS or disease subtype.”

Simon C. Heath^1,2 and Daniel E. Weeks³

¹ CNAG-CRG, Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology (BIST), Baldiri i Reixac 4, 08028 Barcelona, Spain

² Universitat Pompeu Fabra (UPF), Barcelona, Spain

³ Departments of Human Genetics and Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15217 USA. weeks@pitt.edu

References

Z. Wang, A.D. Sadovnick, A.L. Traboulsee, J.P. Ross, C.Q. Bernales, M. Encarnacion, I.M. Yee, M. de Lemos, T. Greenwood, J.D. Lee, et al. Neuron, 90 (2016), pp. 948–954

On 2016 Aug 08, Daniel Weeks commented:

We raise additional concerns in a comment posted in the 'Comments' section of Neuron's web page for this paper, accessible via this link: http://www.cell.com/neuron/comments/S0896-6273(16)30126-X. These additional concerns center on questionable assumptions in Wang et al's LOD score computations as well as miscalculated p-values in their Table 1.

Simon Heath^1,2 and Daniel E. Weeks³

¹ CNAG-CRG, Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology (BIST), Baldiri i Reixac 4, 08028 Barcelona, Spain

² Universitat Pompeu Fabra (UPF), Barcelona, Spain

³ Departments of Human Genetics and Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15217 USA

On 2016 Oct 21, Carles Vilarino-Guell commented:

Case-Control Studies Are Not Familial Studies (2016) Neuron; Volume 92, Issue 2, p339–341.

http://www.cell.com/neuron/fulltext/S0896-6273(16)30695-X

On 2016 Jun 29, Chris Cotsapas commented:

We sought to validate the association of rs61731956 with MS susceptibility in our ongoing study of low-frequency missense variation in MS. After stringent quality control, we used linear mixed models to meta-analyze 32,852 cases and 36,538 controls of European ancestry in 14 country-level strata, genotyped for 250,000 low-frequency non-synonymous variants across all exons using Illumina's HumanCore Exome array. We detected the minor allele rs61731956-A in nine of our strata, but find no evidence of association with overall MS risk (meta-analysis beta = 0.06, p = 0.32). As Wang et al report this association specifically with PPMS, we compared 1,399 PPMS cases to 13,537 RRMS cases directly in five strata with available clinical course information, and also find no evidence of association with disease subtype (meta-analysis beta = 2.35, p = 0.39). Our previous linkage analysis of >700 multiplex families [Sawcer S, 2005] further supports this conclusion (multipoint LOD = 0.0), as does earlier work from the Canadian Collaborative Project on the Genetic Susceptibility to Multiple Sclerosis (CCPGSMS) with no evidence of linkage in 40 Canadian families with four or more affected individuals, authored by members of the Wang et al study team [Willer CJ, 2007].

Based on their interpretation of segregation patterns for rs61731956, Wang et al go on to genotype common variants in the NR1H3 locus in 2,053 MS patients and 799 healthy controls, but fail to detect any association with overall MS risk. They then report that four of their familial cases have a clinical course consistent with that of primary progressive MS (PPMS), and perform a secondary, stratified analysis of clinical course with the five tagging SNPs. They reduce their sample size to 420 PPMS and 1,287 RRMS patients for whom clinical course information was available, and describe an association between rs2279238 (OR = 1.35, p = 0.001) and PPMS, but not RRMS, risk. The IMSGC has already reported a disease risk association in this region [International Multiple Sclerosis Genetics Consortium (IMSGC)., 2013] based on 14,498 MS cases and 24,091 controls to rs7120737, which is 420kb away and in moderate LD with rs2279238 (r² = 0.62, D'= 0.82 in the 1000 Genomes CEU panel). We also genotyped rs3824866, a perfect proxy for rs2279238 (r² = 1, D' = 1 in the 1000 Genomes CEU panel), which shows modest association to MS risk (p = 2.1 x 10^-5). Conditioning on rs7120737 fully explains this association, indicating that the result reported by Wang et al is a modest proxy for the strong signal we have previously reported.

Our 13-fold larger dataset therefore supports a more conventional interpretation of the data presented by Wang et al: there is no association between the low frequency NR1H3 p.Arg415Gln variant rs61731956 and MS risk, but a common haplotype spanning the NR1H3 locus is associated with overall MS susceptibility, despite the failure of Wang et al to detect it in their modestly sized cohort. Our data does not support an association specific to clinical course or PPMS. The false positive likely arose because Wang et al base their conclusions on a total of four affected carriers of the variant, and contravene standard practice by analyzing only five polymorphisms in the NR1H3 locus, not controlling for population stratification, and failing to meet rigorous thresholds of significance for common variation (p < 5 x 10^-8) or for family-based linkage (LOD > 3) [Kruglyak L, 1995; Altshuler D, 2008].

Beyond these technical issues, Wang et al appear to have succumbed to an error in logic in their analysis. Although individually rare, coding variants are exceedingly common in the population: of 7,404,909 variants identified by the Exome Aggregation Consortium in 60,706 individuals, 99% have a minor allele frequency of <1% and 54% are seen exactly once in those data. Therefore, there is complete certainty of observing at least one such variant in two closely related individuals, as Wang et al have done. This is reinforced by their observation of rs61731956-A in multiple unaffected individuals and the presence of this variant in 21/60,706 unselected ExAC individuals. This is not a unique false positive finding, as previous studies of equivalently small sample size have reported MS risk associations to low-frequency coding variants in CYP27B1 [Ramagopalan SV, 2011] and SAIE [Surolia I, 2010], with both results failing to replicate in much larger studies with adequate statistical power [Hunt KA, 2011; Ban M, 2013; Barizzone N, 2013].

We note that the experimental demonstration that p.Arg415Gln alters the heterodimerization efficiency between the NR1H3 product liver X receptor alpha and the retinoid X receptor alpha has no bearing on the association to MS pathogenesis. Many non-synonymous variants have dramatic effects on protein function, but in the absence of robust association to disease this alone cannot support a pathogenic argument [MacArthur DG, 2014].

The combination of our negative results from a 13-fold larger dataset and the methodological and logical flaws in the work presented by Wang et al categorically refute the bold claim that NR1H3 variation defines a Mendelian subtype of MS, which would be the first monogenic form of the disease ever described. Such a discovery would have enormous implications for diagnosis of a subset of cases, prognosis and genetic counseling of extended family members, and eventually for clinical management of the disease in carriers. Unfortunately, the evidence provided by Wang et al does not support this conclusion.

A more complete version of this response, including data tables, is available on biorxiv.

Chris Cotsapas, Yale School of Medicine and Broad Institute of Harvard and MIT, on behalf of the IMSGC.

On date unavailable, commented:

None

On 2016 Jun 27, Daniel MacArthur commented:

We thank the authors for their reply, and for their comparison of their results with those for the G2019S variant in LRRK2 and its association with Parkinson's disease. As a well-established and well-studied pathogenic variant, LRRK2 G2019S is indeed an excellent example to illustrate two points:

1. Genuinely pathogenic variants are enriched in cases over controls. In 23andMe's cohort, the largest dataset for which we were able to find published allele counts, LRRK2 G2019S has an allele frequency of 0.09% in controls and 1.1% in cases, with an odds ratio of 9.6 (Do CB, 2011, see Table S1) for allele frequency breakdown by case/control status]. Another study of Europeans found an allele frequency of 0.8% in idiopathic PD cases (Gilks WP, 2005). The allele frequency in ExAC is accordingly 10-fold lower, 0.04% overall and 0.06% among non-Finnish Europeans. Among PD cases with a positive family history, the variant exhibits even stronger enrichment, with allele frequencies on the order of a few percent (Kachergus J, 2005, Nichols WC, 2005, Di Fonzo A, 2005). Results from other populations are in agreement: among North African Arabs and Ashkenazi Jews, the variant has an allele frequency of 0.6 - 1.4% in the general population, but a frequency of 9 - 19% in Parkinson's disease cases (Lesage S, 2006, Ozelius LJ, 2006).

2. Pathogenic variants may be found in ExAC, but at a frequency consistent with disease prevalence, penetrance, and allelic heterogeneity. Published estimates of lifetime risk of Parkinson's disease range from 3.7% (Elbaz A, 2002) to 6.7% (Driver JA, 2009). LRRK2 G2019S accounts for ~1% of all cases (as noted above), and is estimated to confer lifetime risk of ~32% (Ozelius LJ, 2006, Goldwurm S, 2007). From these figures, one can infer that allele frequencies even as high as 6.7% * 1% / 32% = 0.2% (a few times higher than what we see in ExAC) would not be surprising for this variant.

On both accounts, LRRK2 G2019S stands in contrast to NR1H3 R415Q. NR1H3 R415Q has no evidence of any enrichment in MS cases compared to population controls, and its allele frequency in ExAC is inconsistent with any but the lowest (<2%) penetrance when one considers that multiple sclerosis is less common than Parkinson's disease, and that this variant is found in only a small minority of cases. This contrast thus simply highlights the inadequacy of the genetic evidence supporting any role of NR1H3 R415Q in the etiology of MS.

Eric Minikel and Daniel MacArthur, Broad Institute of MIT and Harvard

On 2016 Jun 17, Carles Vilarino-Guell commented:

We read Mr. Minikel and Dr. MacArthur’s commentary with interest; although we respect their opinion, we are disappointed to see that it is solely based on two pieces of information. We would encourage anyone interested in the subject to read the whole article, assess all the scientific evidence with an open mind, and reach their own informed conclusion.

Regarding the two points of controversy, we would just like to highlight that in common complex neurological disorders it is not unusual for pathogenic mutations to have reduced penetrance, and to be present in the ExAC database. For example, in Parkinson’s disease (once considered to have no genetic component) widely variable penetrance estimates have been described (Trinh et al. 2014); and the LRRK2 G2019S mutation, which is irrefutably pathogenic, can be found in 47 individuals from the ExAC collection (rs34637584).

On 2016 Jun 13, Daniel MacArthur commented:

This paper reports that an NR1H3 variant, rs61731956, encoding p.Arg415Gln, causes familial multiple sclerosis (MS) (Wang Z, 2016). We have some major concerns about the evidence for the effect of the R415Q variant on risk for MS, which rests on two pedigrees with imperfect segregation with disease.

The reported data allows us to provide an estimate of penetrance for the R415Q variant. This variant was found in 1 out of 2053 individuals in a multiple sclerosis case series, but is also seen (as the authors note in passing) in 21 individuals among the 60,706 present in the Exome Aggregation Consortium (ExAC) collection (11-47290147-G-A). Enrichment in cases over controls is one important criterion for establishing pathogenicity of sequence variants (MacArthur DG, 2014, Richards S, 2015).

The ancestry distribution of the case series reported in Wang Z, 2016 is not stated, but the series was collected in Canada and appears to be of predominantly European ancestry (Sadovnick AD, 1998, Traboulsee AL, 2014). The 21 individuals with this variant in ExAC are all of non-Finnish European ancestry, with 66,738 non-Finnish European chromosomes having genotype calls for this variant. Thus, the variant is not significantly enriched in cases over ExAC population controls (P = .56) - indeed, its allele frequency is lower in MS cases (0.02%) than in ExAC European population controls (0.03%).

A review of lifetime risk estimates for MS found the best estimates of lifetime risk of MS to be 0.25% for women and 0.14% for men (Alonso A, 2008, see Table 1). Using the allele frequencies observed in ExAC and in the case series, along with these estimates of lifetime risk, we can apply formulae for calculating the penetrance or lifetime risk, and confidence intervals thereof, for reportedly Mendelian variants as described in (Kirov G, 2014, Minikel EV, 2016). The upper bound of the 95% confidence interval is 1.7% for women and 0.9% for men, indicating that this variant contributes extremely weakly, if at all, to MS risk.

The functional characterization of the effects of the p.Arg415Gln variant on gene function, while potentially interesting, does not provide independent support for a role of this gene in MS risk.

We urge the community to consider rigorous statistical approaches and independent replication before making strong claims of pathogenicity. In this case, publicly available data (and indeed data that are actually noted in the paper) are sufficient to strongly suggest that this variant has little or no effect on MS risk. Independent analyses of this variant in large case-control studies of MS are needed, and we look forward to seeing the results of such analyses from the MS community in the near future.

Eric Vallabh Minikel and Daniel MacArthur, Broad Institute of MIT and Harvard

On 2016 Jun 13, Daniel MacArthur commented:

This paper reports that an NR1H3 variant, rs61731956, encoding p.Arg415Gln, causes familial multiple sclerosis (MS) (Wang Z, 2016). We have some major concerns about the evidence for the effect of the R415Q variant on risk for MS, which rests on two pedigrees with imperfect segregation with disease.

The reported data allows us to provide an estimate of penetrance for the R415Q variant. This variant was found in 1 out of 2053 individuals in a multiple sclerosis case series, but is also seen (as the authors note in passing) in 21 individuals among the 60,706 present in the Exome Aggregation Consortium (ExAC) collection (11-47290147-G-A). Enrichment in cases over controls is one important criterion for establishing pathogenicity of sequence variants (MacArthur DG, 2014, Richards S, 2015).

The ancestry distribution of the case series reported in Wang Z, 2016 is not stated, but the series was collected in Canada and appears to be of predominantly European ancestry (Sadovnick AD, 1998, Traboulsee AL, 2014). The 21 individuals with this variant in ExAC are all of non-Finnish European ancestry, with 66,738 non-Finnish European chromosomes having genotype calls for this variant. Thus, the variant is not significantly enriched in cases over ExAC population controls (P = .56) - indeed, its allele frequency is lower in MS cases (0.02%) than in ExAC European population controls (0.03%).

A review of lifetime risk estimates for MS found the best estimates of lifetime risk of MS to be 0.25% for women and 0.14% for men (Alonso A, 2008, see Table 1). Using the allele frequencies observed in ExAC and in the case series, along with these estimates of lifetime risk, we can apply formulae for calculating the penetrance or lifetime risk, and confidence intervals thereof, for reportedly Mendelian variants as described in (Kirov G, 2014, Minikel EV, 2016). The upper bound of the 95% confidence interval is 1.7% for women and 0.9% for men, indicating that this variant contributes extremely weakly, if at all, to MS risk.

The functional characterization of the effects of the p.Arg415Gln variant on gene function, while potentially interesting, does not provide independent support for a role of this gene in MS risk.

We urge the community to consider rigorous statistical approaches and independent replication before making strong claims of pathogenicity. In this case, publicly available data (and indeed data that are actually noted in the paper) are sufficient to strongly suggest that this variant has little or no effect on MS risk. Independent analyses of this variant in large case-control studies of MS are needed, and we look forward to seeing the results of such analyses from the MS community in the near future.

Eric Vallabh Minikel and Daniel MacArthur, Broad Institute of MIT and Harvard

On 2016 Jun 29, Chris Cotsapas commented:

We sought to validate the association of rs61731956 with MS susceptibility in our ongoing study of low-frequency missense variation in MS. After stringent quality control, we used linear mixed models to meta-analyze 32,852 cases and 36,538 controls of European ancestry in 14 country-level strata, genotyped for 250,000 low-frequency non-synonymous variants across all exons using Illumina's HumanCore Exome array. We detected the minor allele rs61731956-A in nine of our strata, but find no evidence of association with overall MS risk (meta-analysis beta = 0.06, p = 0.32). As Wang et al report this association specifically with PPMS, we compared 1,399 PPMS cases to 13,537 RRMS cases directly in five strata with available clinical course information, and also find no evidence of association with disease subtype (meta-analysis beta = 2.35, p = 0.39). Our previous linkage analysis of >700 multiplex families [Sawcer S, 2005] further supports this conclusion (multipoint LOD = 0.0), as does earlier work from the Canadian Collaborative Project on the Genetic Susceptibility to Multiple Sclerosis (CCPGSMS) with no evidence of linkage in 40 Canadian families with four or more affected individuals, authored by members of the Wang et al study team [Willer CJ, 2007].

Based on their interpretation of segregation patterns for rs61731956, Wang et al go on to genotype common variants in the NR1H3 locus in 2,053 MS patients and 799 healthy controls, but fail to detect any association with overall MS risk. They then report that four of their familial cases have a clinical course consistent with that of primary progressive MS (PPMS), and perform a secondary, stratified analysis of clinical course with the five tagging SNPs. They reduce their sample size to 420 PPMS and 1,287 RRMS patients for whom clinical course information was available, and describe an association between rs2279238 (OR = 1.35, p = 0.001) and PPMS, but not RRMS, risk. The IMSGC has already reported a disease risk association in this region [International Multiple Sclerosis Genetics Consortium (IMSGC)., 2013] based on 14,498 MS cases and 24,091 controls to rs7120737, which is 420kb away and in moderate LD with rs2279238 (r² = 0.62, D'= 0.82 in the 1000 Genomes CEU panel). We also genotyped rs3824866, a perfect proxy for rs2279238 (r² = 1, D' = 1 in the 1000 Genomes CEU panel), which shows modest association to MS risk (p = 2.1 x 10^-5). Conditioning on rs7120737 fully explains this association, indicating that the result reported by Wang et al is a modest proxy for the strong signal we have previously reported.

Our 13-fold larger dataset therefore supports a more conventional interpretation of the data presented by Wang et al: there is no association between the low frequency NR1H3 p.Arg415Gln variant rs61731956 and MS risk, but a common haplotype spanning the NR1H3 locus is associated with overall MS susceptibility, despite the failure of Wang et al to detect it in their modestly sized cohort. Our data does not support an association specific to clinical course or PPMS. The false positive likely arose because Wang et al base their conclusions on a total of four affected carriers of the variant, and contravene standard practice by analyzing only five polymorphisms in the NR1H3 locus, not controlling for population stratification, and failing to meet rigorous thresholds of significance for common variation (p < 5 x 10^-8) or for family-based linkage (LOD > 3) [Kruglyak L, 1995; Altshuler D, 2008].

Beyond these technical issues, Wang et al appear to have succumbed to an error in logic in their analysis. Although individually rare, coding variants are exceedingly common in the population: of 7,404,909 variants identified by the Exome Aggregation Consortium in 60,706 individuals, 99% have a minor allele frequency of <1% and 54% are seen exactly once in those data. Therefore, there is complete certainty of observing at least one such variant in two closely related individuals, as Wang et al have done. This is reinforced by their observation of rs61731956-A in multiple unaffected individuals and the presence of this variant in 21/60,706 unselected ExAC individuals. This is not a unique false positive finding, as previous studies of equivalently small sample size have reported MS risk associations to low-frequency coding variants in CYP27B1 [Ramagopalan SV, 2011] and SAIE [Surolia I, 2010], with both results failing to replicate in much larger studies with adequate statistical power [Hunt KA, 2011; Ban M, 2013; Barizzone N, 2013].

We note that the experimental demonstration that p.Arg415Gln alters the heterodimerization efficiency between the NR1H3 product liver X receptor alpha and the retinoid X receptor alpha has no bearing on the association to MS pathogenesis. Many non-synonymous variants have dramatic effects on protein function, but in the absence of robust association to disease this alone cannot support a pathogenic argument [MacArthur DG, 2014].

The combination of our negative results from a 13-fold larger dataset and the methodological and logical flaws in the work presented by Wang et al categorically refute the bold claim that NR1H3 variation defines a Mendelian subtype of MS, which would be the first monogenic form of the disease ever described. Such a discovery would have enormous implications for diagnosis of a subset of cases, prognosis and genetic counseling of extended family members, and eventually for clinical management of the disease in carriers. Unfortunately, the evidence provided by Wang et al does not support this conclusion.

A more complete version of this response, including data tables, is available on biorxiv.

Chris Cotsapas, Yale School of Medicine and Broad Institute of Harvard and MIT, on behalf of the IMSGC.

On 2016 Aug 08, Daniel Weeks commented:

We raise additional concerns in a comment posted in the 'Comments' section of Neuron's web page for this paper, accessible via this link: http://www.cell.com/neuron/comments/S0896-6273(16)30126-X. These additional concerns center on questionable assumptions in Wang et al's LOD score computations as well as miscalculated p-values in their Table 1.

Simon Heath^1,2 and Daniel E. Weeks³

¹ CNAG-CRG, Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology (BIST), Baldiri i Reixac 4, 08028 Barcelona, Spain

² Universitat Pompeu Fabra (UPF), Barcelona, Spain

³ Departments of Human Genetics and Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15217 USA