the causal relationship may be negative, we say, but there is an uncontrolled variance making the apparent relationship positive.
The logic of p value may lead us accept a positive relation in fact it is a negative relation.
與貝氏邏輯相比，p值的邏輯會讓研究者越來越確信與真正事實相反的證據為真。如同Simpson's paradox的狀況。
it does not need to involve a plausible alternative
This one-way skepticism can not reveal which inference would be true when we falsified the recent one.
單向懷疑論能引導人類智慧發現證據的侷限與錯誤，卻無法指出真正有效的證據是什麼。
Don’t believe everything you see.
Richard Morey attempted to argue the helplessness of this one-way skepticism to the accumulation of scientific knowledge.
這篇網誌的論述立場成立在「單向」懷疑論的無用性。
A person who believed everything they believed on the internet could be misled. Put another way, their standard of evidence (“it was on the internet”) is too weak. This is not an unfamiliar line of reasoning.
Another inference standard people hardly agree in the daily life. 如果個人不能完全相信網路的訊息，也不該完全相信p值代表推論的有效性。
it is wrong to consider a p value as yielding an inference. It is better to think of it as affording critique of potential inferences.
Richar Morey's theme. 論述立場：p值並不能表達推論是否有效，充其量是指出有可能成真的推論。
Event Segmentation Theory
Could EST be comparable with the situational model such as Event Indexing Model (Zwaan et al., 1995)?
a segment of time at a given location that is conceived by an observer tohave a beginning and an end
Authors' definition about event segments.
Kenneth J. Gergen's question: "How does your theoretical approach add value?"
Sarah Durston's comment on materialism: "is it right that all of human experience can be understood as arising from brain function."
Iris' responds (48:20):
Post-talk note:
Total length: 1:20:55; official starting: 11:45
Psychology is like a brick wall. The bricks are effects and theories. Crisis revealed the weakness inside the wall. Started at classical effects... effects are cool and interesting, but effects can not tell us how capacities work. We need theories to explain and predict.
Obstacle 1: Mistaken ideas that theory is built on effects. Theories were from random generated sentences.
Obstacle 2: Lacking an approach for building theories. Iris suggested AI/computer science and philosophy for building the theories in cognitive psychology and other social sciences.
Levels of explanation: What - how1 - how2
Developing theory is thinking
Tools for thinking have paper and pencil, models & formalisms, philosophical analysis, computability & complexity analysis, methamatics & proofs, computer simulations
*Iris' team works on computability and computer simulations.
List them in codebook form.
upload your self testing data on PsyToolkit.
This code is sufficient.
Give the detailed description on this part.
Inconsistency with replication: participants
Error rate of lures and distraction Accuracy of Fillers
Properties of words to be recognized: Lures (Critical words of memory lists) Distraction (Elementary words out of memory lists) Fillers (Elementary words in memory lists)
effect size > 1.5; Optional stop by Bayes Factor
1.5 minutes?
原始研究創造刺激列表的說明
樣本數評估資訊
要保持谦逊：兼容性评估的前提是用于计算区间的统计假设是正确的
應翻為確認統計假設的正確性。這點看出他們的立論基於估計的參數，而非實在的科學理論。統計假設是科學理論推理的延伸，只用推理合乎有效的邏輯形式，有效結果與無效結果都會是科學理論的證據。
和0.05的P值阈值一样，95%置信区间中的95%本身也是约定俗成的。其基础观点就存在问题，即计算出的区间有95％的可能性包含真值；并且95%这个数字让人有一种模糊的感觉——根据这个区间可以得出可靠自信的结论。
前兩點立論基礎薄弱，出現自打嘴巴的論點。
在给定假设的情况下，区间内数值与研究数据的兼容性并不完全相同
原文“not all values inside are equally compatible with the data, given the assumptions. ” 這裡的assumption是指估計的參數，還是科學理論對現實狀況的預測，並沒有明確說明。如果是估計的參數，Amrhein等人也許將P(D|theta)當成P(theta)。
在给定假设的情况下，区间覆盖了与研究数据最兼容的数值，并不意味着区间外的数值一定与研究数据不兼容，只是兼容性不那么高
Amrhein等人並沒有區別他們所談的compatibility是指P(D|theta)還是P(theta|D)
兼容区间
陳瑞麟老師建議“可容性區間”
我们看到了大量具有“统计学显著性”的结果；而不具有“统计学显著性”的结果则被显著低估
豈止低估。不顯著的研究結果經常被鎖起來不見天日。
在置信区间包含风险显著增高的情况下，仅因为结果不具有统计学显著性就推论药物与房颤发生“无关”十分可笑；据此就认为前后两项研究矛盾——即便风险比完全一致——同样非常荒谬。这些常见情况表明我们依赖的统计学显著性阈值有可能误导我们。
Amrhein 等人以此例子顯示confidence interval能突顯不一致的研究之間，評估要測量的效應其實一致的資訊。
Merely reporting likelihood ratios does not produce meaningful control of errors; nor do likelihood ratios mean the same thing in different contexts. So N-P consider the probability distribution of Λ(X), and they want to ensure the probability of the event {Λ(X) ≥ kα} is sufficiently small under H0
NP started design the hypothesis testing from the likelihood ratios, a.k.a. Bayes Factor.
social network information from the general pop-ulation of psychologists
Third method: "a network scale-up estimate, where the general population (within research psychology) will be asked how many people they know who have used QRPs in the last 12 months."
the unmatchedcount technique
Second method: "an unmatched count estimate, where population size will be measured indirectly via item counts."
We simply asked researchers toreport their own QRP use.
First method: "a direct estimate, where participants are asked directly if they have used a QRP within the last 12 months."
it addressed the larger issue of “prevalence”, by definingbehaviors performed within a specified time period.
Second goal of this study. This study investigated the QRPs happened in the particular timing.
replication crisis
First goal of this study.
47.9%
Third summary of QRPs. Italian psychologists.
Fiedler and Schwarz (2016) found less than10% prevalence
Second summary of QRPs survey. German psychologists
John et al. (2012) found 63%
First summary of QRPs survey. American psychologists.
Questionable Research Practices of interest with examples
Included 9 QRPs of John et al.(2012).
The software you develop is part of the methods section, and it is the easiest part to share.
This should be the core principle in every course of research method.
others can learn from what we have done, and revealing how you carried out an experiment is at the heart of any publication.
Every workshop for scientists should teach the latest open methodology.
We should consider the data to be the main publication, and the paper a secondary, less important part
I'm imaging there will be no human language as the primary publication language; or every human language is the primary language to express the scientific knowledge.
imagine if open science was considered normal, and closed science considered weird. Wouldn’t the world be a better place?
In every aspect of science, making "Open" mainstream.