I suppose this formula was in the PAP, but still… is there a good reason why we calculate this as a share of disagreements rather than as a share of all interactions?; any chance this could give us significance when evaluating the effect of more /less heterogenous cases?

Just to reiterate, this table worries me.

again, may be missing smth: here the paper says that the pairwise vialation is biased downward, so the true pariwise violation needs to be higher; but appendix A shows that the bias-corrected pairwise violation is way smaller... also if that was the case, how do you thinkof it, it looks like a huge change and we need to reflect that no?

My sense is that readers may have some concerns about the leave-one-out estimates. How about expanding Appendix B.2 to replicate the analysis in Section 4.2 using those estimates? We could also discuss how the leave-one-out eastimates differ from our main specification and whether the difference between the two can be interpreted as an upper bound (not sure) on the bias. But if it does and the bound is small, that would be reassuring. we can extend it directly in 4.2 perhaps..

in terms of terminology we alter between multidimentisonal decisison and multiple treatment... is that ok?

For example, a difference of 0.05 in these cross-weighted mean custody effects would shift the incarceration estimate by about 0.005. Whether propensity controls adequately isolate the incarceration effect therefore depends on how much custody effects differ between these two sets of cases.

rewrite (make sure i didnt twist the meaning): Using the weights of Bhuller and Sigstad (2024), we find that contamination of the incarceration estimate equals 0.104 times the difference in cross-weighted mean custody effects between cases that prosecutors with a higher propensity to incarcerate are more likely to assign a prison sentence—suspended or unsuspended—and those they are less likely to assign such a sentence, after controlling for their overall custody propensity.

i am not sure with this, but i tried my version how to expalin it: "Intuitively, two prosecutors with the same propensity to recommend custody (any prison term, whether suspended or unsuspended) may recommend custody for different cases. Comparing the prosecutor with the higher incarceration rate to the one with the lower rate therefore changes which cases receive custody, even though the overall custody rate is the same. If the effect of suspended prison sentence relative to non-custodial punishment differs between the cases moved into and out of custody, these effects do not cancel, contaminating the estimated effect of incarceration."

When examiner decisions are multi-dimensional, controlling for non-focal treatment propensities need not isolate the effect of interest.

rewrite the end of this paragrph: This difference is consistent with case heterogeneity contributing to violations, but it is imprecisely estimated. Violations also persist within the homogeneous pool, suggesting that restricting comparisons to similar crime types does not eliminate them.

rewrite: We estimate pairwise monotonicity violation rates of 10.5% on the incarceration margin and 14.0% on incarceration length, within the 10–20% range Sigstad (2026) documents for judicial panels. As the number of prosecutors increases, so does the number of pairs that can violate the case-level IA condition; a reversal in any one pair is enough for the condition to fail. In our sample of 189 prosecutors, it fails in 93.8% of group–case combinations. The implications for 2SLS bias, however, depend on the magnitude of negative weights and treatment-effect heterogeneity, rather than the frequency of violations alone. Although 6.3% of group–case combinations receive negative weights, their combined mass is small, implying negligible bias under our illustrative heterogeneity benchmark.

this should be a new paragrph: also slightly adjsuted wording:

Our data come from an online survey of 189 Polish public prosecutors recruited through regional prosecutors’ offices. Each prosecutor decides four criminal cases presented as written vignettes drawn from a set of twelve. The vignettes were developed with criminal lawyers and a sitting Polish judge and reviewed by a practising Polish prosecutor. Guilt and the legal classification of the offence are stipulated, and prosecutors recommend a sentence type and its severity. Prosecutors are randomly assigned to one of four groups that vary in case homogeneity—from four shoplifting cases to a mix of diverse offences—allowing us to examine the extent to which case heterogeneity contributes to monotonicity violations.

rewrite and shorten: Bhuller and Sigstad (2024) derive conditions under which 2SLS with these controls assigns non-negative weights to the treatment effects of interest and zero weights to effects on other treatment margins. These conditions need not hold even when overall treatment propensities are uncorrelated. By observing prosecutors’ decisions on identical cases, we can calculate the case-level weights directly and assess whether these conditions hold.

these controls (?)

rewrite and shorten: It also lets us examine the common practice of controlling for non-focal treatment propensities to isolate the effect of one decision margin, addressing the broader challenge of identifying multiple treatment effects with instruments under heterogeneous effects (Mueller-Smith, 2015; Heckman et al., 2008; Kirkeboen et al., 2016; Mountjoy, 2022; Humphries et al., 2025).

rewirte: decision

rewrite: controlling for non-focal treatment propensities to isolate

rewirite: complementing Sigstad’s (2026) evidence in a setting without panel dynamics.

delete? not sure what this "signle" refer to?

delete "it"

rewrite: Sigstad (2026) provides the first large-sample estimates of monotonicity violations using judicial panels, where multiple judges vote on the same case. Across five court settings, Imbens–Angrist (IA) monotonicity is violated in 35–49% of non-unanimous cases, while the weaker average-monotonicity condition is violated much less often (4–11%), with small implied biases in 2SLS estimates. Panel decisions, however, may reflect interactions among judges through deliberation, dissent aversion, and unanimity norms. The observed violations may therefore differ from those arising in examiner IV settings where decisions are made by individual examiners rather than panels. Moreover, the binary votes observed in these panels do not capture the multi-dimensional nature of examiner decisions, whose implications for identification have received increasing attention (Andresen and Huber, 2021; Bhuller and Sigstad, 2024; Humphries et al., 2025).

rewrite:A typical example is multi-dimensional examiner decisions: when the researcher instruments for only one decision margin, the other dimensions may also affect the outcome, violating exclusion. A standard response is to control for the examiner’s propensities on these other dimensions (Mueller-Smith, 2015; Humphries et al., 2025).

rewrite: propensity controls isolate the effect of interest

rewrite: The exclusion restriction is also difficult to assess, so applied work often adjusts the specification to address plausible sources of violation.

Monotonicity is harder to assess directly because doing so requires observing multiple examiners deciding the same case, which administrative data rarely allow.

rewrite: The first two assumptions are routinely assessed using balance tests and first-stage diagnostics.

i dont like using stricter examiners w/o an introducinng what it means, as for some of the examiners "stricker" may not mean anything no? how about smth like: "monotonicity, requiring examiners’ treatment decisions to follow a common ordering across cases."

"propensities" doenst sound enough; propensities to assign treatment?? // treatment propensitites?

rewrite: an exclusion restriction requiring examiners to affect outcomes only through the treatment of interest

rewrite: ...we refer to this approach as...

rewrite: ...to identify the causal effects of their decisions...

strictly speaking we promise 95% ci and then report standard errors

what are these? CIs? does not sound right

a) I’ll send comments on the figure in a separate email. b) Table 7 says that he only average-monotonicity violation is in Group 2, which is inconsistent with this figure.

what is the ranking here?

comaprable to the results from section 4.4

lets use consistent vignette labels across both figures. Either adopt the labels from Figure 1 or, more informatively, label the shoplifting vignettes by their case numbers so readers can see that Case 5 is identical across all groups and that Case 7 is also the same for vignette wherever it appears. BTW: around 100 prosecutors decided both Cases 5 and 7. if we needed a larger sample of decisions on the same two cases for any of the robustness exercises?

proposed paragraph: The prevalence of monotonicity violations depends on both the number of examiners and the dispersion of their stringencies. As the examiner pool grows, so does the number of pairs that can violate IA monotonicity; a reversal in any one pair is enough for the condition to fail. Average monotonicity may instead become easier to satisfy: reversals between some pairs can be offset by disagreements in the expected direction between others. Greater stringency dispersion leaves less scope for pairwise violations. At the extreme, an examiner who never assigns treatment and one who always does cannot violate monotonicity, whereas examiners with similar treatment rates may disagree substantially about which cases warrant treatment. We next examine how the monotonicity measures vary with the number of prosecutors and the dispersion of their stringencies.

Monotonicity by Prosecutor Pool Size and Stringency Dispersion (Q2)

delete; or move to note?

not sure it is clear what the commaprison is; 0.8-6 percetngae points bias for what effecT? i may be missing something here

Following Sigstad (2026), we assess the potential bias from negative 2SLS weights by assuming that treatment effects are twice as large for negatively weighted cases (“defier” cases) as for positively weighted cases (“complier” cases). This is an illustrative scenario, not an upper bound: a larger gap in treatment effects would increase the bias proportionally. Let \(W^{+}\) denote the total positive weight mass and \(W^{-}\) the absolute value of the total negative weight mass. The asymptotic bias of 2SLS relative to the effect among positively weighted cases, \(\tau_C\), is

strictly speking this is not the case; one of the shoplifthing is decided by all prosecuotrs; and one by prosectuos form two groups... so probalby adjust this sentence

"Headline" is probably not informative

I may not have understood your Journal of Econometrics paper correctly, so please correct me if I’m mixing things up. Here we are using the ordered specification—any prison sentence and unsuspended imprisonment—right? and the unordered specifciaiton is in the appendix table T12 right? is there a good motivaiton for haveing the ordered specification in the main text and the unordrede in the appendix, I can imageine many criminoogics would oppose the ordered version.

I find this potentially problematic: what do you mean by saying there is no sampling uncertainty? The weights are deterministic conditional on the observed data, but they would change with a different sample of prosecutors. Also, the PAP explicitly specifies bootstrap confidence intervals for Q1–Q5, including the weight summaries in Q3.

this framing doesnt seem consist with what we know about judges (proscutores) behavior; more geneeral intro:

Examiners often choose among several alternatives rather than make a binary decision. For example, prosecutors and judges choose among sentence types, with the available options depending on the jurisdiction and the circumstances of the case. Alongside unsuspended imprisonment, these may include a suspended prison sentence, a fine, or other restrictions on liberty.

delete

a) in section 3.3, we say that the gradient was pre-registered b) the PAP says this: "We additionally report the per-group pairwise violation rate VR𝑔 for each 𝑔 ∈ {2,3,4,5}, tracing the gradient from the most homogeneous(Group2: allshoplifting) through the mostheterogeneous (Group 5: diverse crime types). We report both quantities for the binary and continuous outcomes with BCa 95% bootstrap confidence intervals (Section 5)."

so i would drop the note it wasnt pre-registered

move to the appedix.. I understand the tradeoff, but still marginally prefer to move it to the appendix and dont bother readers of the main text by quesitons we dont answer

isnt this too strong; striclty speaking while not st. significant, it is cca a 50% increase (8 -> 12%); also we dont really test the value of conditioning on case characterisitsc;

my suggestion for this paragraph: The four vignette groups vary in case homogeneity: Groups 1 and 2 contain related crime types, while Groups 3 and 4 span more diverse offences. Table 2 compares the pooled homogeneous and heterogeneous groups. Pairwise violation rates are higher in the heterogeneous pool: 12.4% versus 8.5% for incarceration, a difference of 3.9 percentage points, and 15.4% versus 12.0% for sentence length. These differences are consistent with case heterogeneity contributing to monotonicity violations, but the estimates are imprecise: a one-sided permutation test yields \(p=0.16\) for the binary comparison. Moreover, the comparison captures differences between the selected vignette bundles rather than isolating case heterogeneity from other case characteristics. The results therefore provide suggestive evidence that more homogeneous case mixes may reduce violations. Appendix Figure 3 presents an exploratory comparison of binary violation rates across the four groups, with 95% bootstrap confidence intervals.

Appendix Tables 6 and 7 report the average-monotonicity and 2SLS-weight results using three pre-specified stringency encodings and leave-one-out stringency measures, respectively.

suggestion: make it two paragrephs, give more space for the supplemenatry findings (in line with deleting the end of the third paragraph):

We estimate pairwise monotonicity violation rates of 10.5% on the incarceration margin and 14.0% on incarceration length, within the 10–20 percent range Sigstad (2026) documents for judicial panels. As the number of prosecutors increases, the case-level IA condition, which requires monotonicity across every pair, becomes increasingly likely to be violated. In the full sample of 189 prosecutors, it is violated in nearly every group–case combination (93.8%). The implications for 2SLS bias, however, depend on the total magnitude of negative weights and treatment-effect heterogeneity, rather than the frequency of violations alone. Although 6.3% of group–case combinations receive negative weights, their combined magnitude is small, implying negligible bias under our illustrative treatment-effect assumptions.

Two supplementary analyses examine how these findings vary with prosecutor composition and case mix. First, subsampling shows that negative weight mass is small when there are many prosecutors or their stringencies differ substantially. It becomes appreciable mainly when prosecutors are both few and similar in stringency, in which case the implied bias can be moderate. Second, the binary pairwise violation rate is higher in the heterogeneous case pool than in the homogeneous pool (12.4% versus 8.5%). This difference is consistent with case heterogeneity contributing to violations, but the estimate is too imprecise to establish a clear difference between the two pools.

Drop this here; let’s keep these opening paragraphs focused on the two key questions and discuss the supplementary ones only when presenting the findings.

delete?

my attempt to make it understadable for slow reader as myself:

Following Sigstad (2026), we estimate each prosecutor’s stringency as their incarceration rate across the four cases in their vignette group, \(\hat p_i=K^{-1}\sum_{k=1}^{K}D_{ik}\). We rank prosecutors within each group by this stringency rate, breaking ties with small random noise. For each case, the IA violation indicator equals one if any less stringent prosecutor recommends incarceration while a more stringent prosecutor does not, and zero otherwise. The case-level IA violation rate is the share of cases with at least one such reversal, averaged equally across groups. Note that these measures depend only on prosecutors’ ranking, not on the numerical differences between their stringencies. Average monotonicity instead depends on the case-level weights \(w_k=\operatorname{Cov}i(\hat p_i,D{ik})/\operatorname{Var}i(\hat p_i)\), calculated within each group. A case receives a negative weight when prosecutors who recommend incarceration are, on average, less stringent than those who do not. We report both the share of cases with negative weights and the sum of those weights, averaging each measure equally across groups. The former captures the incidence of average-monotonicity violations; the latter captures their magnitude. Their implications for the 2SLS estimand depend on treatment-effect heterogeneity, as illustrated in Section 4.2. Because these measures depend on numerical stringencies, we assess robustness using three encodings: the incarceration rate, mean incarceration length, and ordinal sentence severity. Our primary statistic is the pairwise violation rate. For each pair of prosecutors \(i\) and \(j\) within the same group, let \(n{ij}=\sum_k D_{ik}(1-D_{jk})\) count cases in which \(i\) recommends incarceration and \(j\) does not. We calculate [ VR= \frac{\sum_{i<j}\min(n_{ij},n_{ji})} {\sum_{i<j}(n_{ij}+n_{ji})} ] within each group and average the resulting rates equally across groups. For each pair, the numerator counts disagreements in the less frequent direction, while the denominator counts all disagreements. Thus, the statistic measures the share of disagreements that run against the pair’s predominant ordering. For binary decisions, this is equivalent to counting reversals of the stringency ranking: \(\hat p_i-\hat p_j=(n_{ij}-n_{ji})/K\). When stringencies are tied, \(n_{ij}=n_{ji}\), so either ordering produces the same number of reversals. The pairwise rate is therefore invariant to the tie-breaking draw.

if we change the section titles we can delete this

jsut to make sure: custodial sentences refer to both unsuspended and suspendend prison sentnece or only to unsuspended; this needs to be clear

delete this sentence?

the pairwise VR numbers do not correspond to figure 3!

We have considerable anecdotal evidence suggesting that prosecutors in Central and Eastern Europe may have as muhch or even more influence over sentencing decisions as judges. We are currently conducting an experiment to investigate this, but we do not yet have credible empirical evidence to support the claim.

same comment as before; dont think this is a) needed b) accurate

again, is preferences the right expression?

proper reference

Since the results appear only in the appendix, I would consider moving the corresponding hypotheses there as well.

i may be missing somth, but if both are close to 0, the gap is 0 it doendnt indicate within.type disagreement, does it?

delete; it is not true and dont think it is needed

delete this; one of the last minute changes was to make sure all vignettes have the same set of options

a main sentence type and its severity

delete;

delete: a) nobnody knows what vignette C is and b) not sure what anchor here means

The vignettes included two distinct shoplifting cases.

We do not need to discuss the original group 1, as it was not part of the relevant data collection. To simplify the presentation, I suggest relabelling the four included groups as groups 1–4 throughout the paper.

rewrite: Each prosecutor was asked to make sentencing decisions in four criminal vignettes drawn from a set of 12. The vignettes were developed in close collaboration with criminal lawyers and a sitting Polish judge, then reviewed by a practising public prosecutor. This process aimed to ensure that the vignettes reflected Polish criminal law and prosecutorial sentencing practice, included all relevant information about the offence, the offender’s criminal history, and aggravating and mitigating circumstances, and elicited nontrivial variation in sentencing decisions. The vignettes were also designed so that homogeneous groups contained cases involving similar offences, while heterogeneous groups contained cases involving more dissimilar offences, with both allowing meaningful variation in sentencing decisions. Each vignette was presented alongside the applicable statutory sentencing range. Prosecutors who did not complete all four vignettes were excluded from the monotonicity analysis.

rewrite: The participants are Polish public prosecutors recruited through regional prosecutors’ offices, which distributed an anonymous Qualtrics survey link to prosecutors. Data collection began in June 2026, and the survey remained open until mid-August; 75% of responses were received within the first two weeks. Participation was voluntary and uncompensated. The study was approved by the Commission for Ethics in Research at the Faculty of Social Sciences, Charles University (submission no. 291). The survey collected prosecutor demographics—gender, years of experience, office level, and district—in addition to the decisions described below.

delete this sentence