While not a typical empirical research article, there is a methodology used for identifying relevant sources for its literature review and analyzing those sources.

Educational design research methodology.

why IPIS/ULULA study is not representative or generalisable

methodological 'problems'

field visits? visit surveys and observations ODK

Mathodology: Combining social science methods and underwater reef surveys.

Statistical significance test evaluating whether a sample is greater than or less than a specific value range. Critical distribution area is two-sided.

The probability of rejecting the null hypothesis when it is true.

Variability for individuals themselves in the sample.

Variability of specific scores for individuals in the sample.

A value on a regular grid in three-dimensional space. In this case, composes the 3-dimensional brain image.

MR imaging frequency that displays CSF as the brightest contrast, white matter as the second brightest contrast, and gray matter as the third brightest contrast. Stronger than T2 MR imaging frequency.

3 Tesla-powered scanner model. In our replication, we will be using a different scanner.

A cross presented to research participants in a perception task with the intent directing the participants' attention to wherever the investigator wants them to look.

The amount of time between the end of one stimulus being presented and the start of another stimulus being presented.

MR imaging frequency that displays gray matter as the brightest contrast, white matter as the second brightest contrast, and CSF matter as the third brightest contrast.

Structure at the lower back of the brain, associated with motor control.

Outermost brain layer, associated with higher order cognitive abilities.

Mulot, M., Segalas, C., Leyrat, C., & Besançon, L. (2021). Re: Subramanian and Kumar. Vaccination rates and COVID-19 cases. European Journal of Epidemiology, 36(12), 1243–1244. https://doi.org/10.1007/s10654-021-00817-6

Important differences between Agile and Waterfall methodology

The waterfall methodology for software development is rapidly losing popularity, while the Agile methodology is increasingly used for software development by companies around the world.

Quite nice that they strengthened their analysis with user studies, something that I haven't seen often in these type of ML papers.

Question: How can you actually define a style coordinate? How should I envision this?

Replicating scientific results is tough—But essential. (2021). Nature, 600(7889), 359–360. https://doi.org/10.1038/d41586-021-03736-4

Tunç, D. U., Tunç, M. N., & Eper, Z. B. (2021). Is Open Science Neoliberal? PsyArXiv. https://doi.org/10.31234/osf.io/ft8dc

Grimmer & Stewart (2013) - Text as Data: The Promise and Pitfalls of Automatic ContentAnalysis Methods for Political Texts

• urn:x-pdf:5bb906b65a58358ba98fbc3ee72836f9
• https://is.gd/rBVTEI

All imaging studies face questions of validity and should (and many do) link to comprehensive details on instrumentation, methodology, and interpretation. Apparently, the professional consensus remains that, properly executed and interpreted, fMRI and other functional imaging techniques based on detection of oxygenation can lead to highly valid conclusions. (See Nautil.us article.)

Kendall's tau-c

a non-parametric measure of relationships between columns of ranked data. The Tau correlation coefficient returns a value of 0 to 1, where: 0 is no relationship, 1 is a perfect relationship. source: https://www.statisticshowto.com/kendalls-tau/)

Control variable (alternative explanations) **Socioeconomic level of residents: When controlling for High socioeconomic factors (such as high education and high income in relationship to median income).

**Region Assumes there may be regional difference between cities, such as how old they are, their state of disrepair)_ Categorized regions northeast, north central, south and west.

Sub: The ability to mobilize community people and resources requires a management class.

Independent Variable: Managers, Proprietors and officals/ to employed labor force. Census data, number of managers

Dependent Variable: Success in Urban Renewal: Stages: Planning, Execution, Completion She classifies cities by: Those that exectued Those the abandoned the problem (for whatever reason) Those who never tried urban renewal - although they quality..

Intervening variables:

Too bad there wasn't more information in the citations, even just the author & title, let alone a short summary. I wouldn't follow the link.

Dealing with power asymmetries.

consent

The Indian Express. “Why We Need to Count the Covid Dead,” July 20, 2021. https://indianexpress.com/article/opinion/columns/india-covid-deaths-second-wave-7412619/.

Logg, Jennifer M., and Charles A. Dorison. “Pre-Registration: Weighing Costs and Benefits for Researchers.” Organizational Behavior and Human Decision Processes 167 (November 1, 2021): 18–27. https://doi.org/10.1016/j.obhdp.2021.05.006.

Metascience 2021. (n.d.). Retrieved June 27, 2021, from https://metascience2021.org/

Mosleh, M., Pennycook, G., & Rand, D. G. (2021). Field experiments on social media [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/dgmc2

Calster, B. V., Wynants, L., Riley, R. D., Smeden, M. van, & Collins, G. S. (2021). Methodology over metrics: Current scientific standards are a disservice to patients and society. Journal of Clinical Epidemiology, 0(0). https://doi.org/10.1016/j.jclinepi.2021.05.018

Baghal, T. A., Wenz, A., Sloan, L., & Jessop, C. (2021). Linking Twitter and survey data: Asymmetry in quantity and its impact. EPJ Data Science, 10(1), 1–20. https://doi.org/10.1140/epjds/s13688-021-00286-7

Reproduction number (R) and growth rate: Methodology—GOV.UK. (n.d.). Retrieved May 13, 2021, from https://www.gov.uk/government/publications/reproduction-number-r-and-growth-rate-methodology/reproduction-number-r-and-growth-rate-methodology

He is detailing his methodology - it’s not random

Untools - Tools for better thinking

Collection of thinking tools and frameworks to help you solve problems, make decisions and understand systems.

See: Weaver, J. A. and N. Snaza (2017). "Against Methodocentrism in Educational Research." Educational Philosophy and Theory 49(11): 1055-1065.

Bauer, B., Larsen, K. L., Caulfield, N., Elder, D., Jordan, S., & Capron, D. (2020). Review of Best Practice Recommendations for Ensuring High Quality Data with Amazon’s Mechanical Turk. PsyArXiv. https://doi.org/10.31234/osf.io/m78sf

This article introduces a special issue of Qualitative Inquiry that focuses on using "concept" as method in the education and social sciences. They describe this exploratory approach where the method emerges during the process of research.

Online Research Tools and Techniques. (2020, September 16). https://www.youtube.com/watch?v=wGWqBtDkOFs

Health Nerd on Twitter. (n.d.). Twitter. Retrieved October 17, 2020, from https://twitter.com/GidMK/status/1316511734115385344

Dr Natalie Shenker on Twitter. (n.d.). Twitter. Retrieved October 13, 2020, from https://twitter.com/DrNShenker/status/1314475759508107265

Kekecs, Z., Szaszi, B., & Aczel, B. (2020). ECO, an expert consensus procedure for developing robust scientific outputs [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/9gqru

Computational Social Science to Address the (Post) COVID-19 Reality. (2020, June 27). https://www.youtube.com/watch?v=7d-Dq0e1JJ0&list=PL9UNgBC7ODr6eZkwB6W0QSzpDs46E8WPN&index=4

Online Research: From Funding to Data Collection. (n.d.). Association for Psychological Science - APS. Retrieved September 25, 2020, from https://www.psychologicalscience.org/news/online-research.html

Laghaie, A., & Otter, T. (2020). Measuring evidence for mediation in the presence of measurement error [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/5bz3f

Fujita, Shigeru, Giuseppe Moscarini, and Fabien Postel-Vinay. ‘Measuring Employer-to-Employer Reallocation’. Working Paper. Working Paper Series. National Bureau of Economic Research, July 2020. https://doi.org/10.3386/w27525.

Coronavirus: The inside story of how UK’s “chaotic” testing regime “broke all the rules.” (n.d.). Sky News. Retrieved July 17, 2020, from https://news.sky.com/story/coronavirus-the-inside-story-of-how-uks-chaotic-testing-regime-broke-all-the-rules-12022566

James Evans—Designing Diversity for Collective Advance (ACM CI’20). (n.d.). Retrieved June 25, 2020, from https://www.youtube.com/watch?v=1OMmJJz0oF0

Puthillam, Arathy. ‘Too WEIRD, Too Fast: Preprints about COVID-19 in the Psychological Sciences’. Preprint. PsyArXiv, 10 June 2020. https://doi.org/10.31234/osf.io/5w7du.

Lanovaz, M., & Turgeon, S. (2020). Tutorial: Applying Machine Learning in Behavioral Research [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/9w6a3

The author also harshly criticizes non-ethnographic qualitative studies solely based on sermons and interviews that try to argue for a individualizing, socially corrosive prosperity Pentecostalism without data that actually documents sociality.

This firing of Snopes managing editor reiterates previous concerns over an unhealthy working environment. This article is old enough I'd like to see followups.

Diminishing your 50/50 partner's contributions & erasing their attributions (accidentally or not) is one of the things I find the most annoying.

Impressive. Her alleged painstaking data-hoarding makes me like her. I'm not sure what to think of David's ambiguity yet.

Etienne's blog! Amazing resource for OSINT; particularly focused on technical attacks.

Comparing survey data (i.e. individual/public opinion) with news coverage (media framing)

Methodology is the critical factor for scientific study, not the result.

I have to wonder at telephone samples of this age group given the propensity of youth to not communicate via voice phone.

"A través de las dificultades de acceso al agua, lo que estaba en juego no era pues solamente una mejora de las condiciones materiales de existencia, sino una lucha por el reconocimiento de su existencia social. Y en la medida en que mi compromiso etnográfico afectaba la dimensión material de las condiciones de vida de los residentes, se inscribía también, de hecho, en la dimensión simbólica de la vida del barrio, revelando así que incluso en las zonas más marginales en apariencia, la vida social tiene que ver con una «doble verdad»16 que el trabajo etnográfico debe aprehender. En este caso preciso, el acceso a la propiedad y a los servicios es el primer momento del acceso a la existencia social, y a su reconocimiento político. " P. 358

https://marcarand.files.wordpress.com/2011/06/bourdieu-la-doble-verdad-del-trabajo.pdf

"Inversamente, los esclarecimientos metodológicos aportados por las otras técnicas de investigación, y la multiplicación de los lugares estudiados, me permitían dirigir otra mirada sobre mi campo etnográfico, y de alguna manera «revisitarlo» (Burawoy, 2010). Retomando el vocabulario de Burawoy, yo pasaba de la «revisita de confirmación» a la «revisita en continuo», pero de tipo «empirista», cuando el investigador descubre dinámicas que superan la comunidad previamente estudiada." P. 356

El enfoque etnográfico efectuado inicialmente me permitía reintegrar, en el momento de la objetivación, el «sentido vivido» por los agentes (Bourdieu, 1980b): me daba cuenta que este me había servido de aquello que Loïc Wacquant llama un «instrumento de deconstrucción de las categorías» (Wacquant, 2008) utilizadas en los enfoques estadísticos. p. 356

The scope and scale of trace data —from settings, public facing fatures, and server-side — presents similar challenges as technological platform changes = new ethics/privacy issues.

What about documenting technological features/artifacts on a stand-alone website or public repository, like Github to account for page limits?

Excellent point and important for my SBTF studies.

I wonder what the methodology of this paper was?

The author makes it clear that she will be explicating her thesis via the example of Emma. In this sentence, she connects Austen's approach to reading, writing, and readership to the notion of literary fiction. Again, I'm not sure if the following paragraphs do live up to the expectation she sets up here.

The author claims here that Emma was inspired by the 1814 novel Discipline by Mary Brunton, which surely is not part of the male literary canon laid out earlier in the article. The author outlines the main themes of Discipline and explains the relationship between the two authors.

I feel like a broken record here, but again, this seems to be a very tenuous point without computational analysis. The author's own language belies this tenuousness as she says that the novel's inspiration "seems to have been a straightforward reaction" to another novel. The word "seems" does not inspire confidence.

This relates to the above annotation. I also agree that people's perceptions of others behaviors are unreliable; but this is an interesting point that in this case, perception may be more important than reality. Of course, that would need to be tested for us to know for sure. I think an interesting future study would be to use SNA and have both the egos and their alters actually track their substance use activities day by day. This would address the perception vs reality issue, as well as the underestimation of own behavior issue. There could still be some social desirability bias though.

This is an excellent use of SNA. I have never seen it used chronologically to map feelings. I love it!

Think about this for your own research. Pay attention to the methods so you can begin to think about how you will collect your own data.

In this brief introduction to the next section of the paper, Murphy challenges existing scholarship that aligns Emma with Shakespeare's A Midsummer Night's Dream. Rather, the author outlines the parallels between Shakespeare's Twelfth Night. I find the connection somewhat tenuous as it almost ignores all of the gender bending and performance of Twelfth Night. While the author's later claim that "the broader themes of deliberate misrepresentation and self-serving delusions" are the tie between the two plays, I find that ignoring the aspects of performance and disguise is problematic.

I also think that this takes away from Murphy's main argument, which is that Austen's view of influence is broader than the historically main canon, evidence by her parody of Brunton's novel. This section seems to show the opposite, which is a parallel between Austen and Shakespeare.

Here, Murphy makes the connection back to readership and the characters of Emma.

This, incidentally, made me think of the quote on the new British ten pound note: "I declare after all there is no enjoyment like reading!" which was certainly a satirical denotation.

This is a key section. Here, the author claims that Jane Austen's Emma is influenced by the rather unremarkable and certainly much less well known novel Discipline. This is in contrast to the existing tradition. Murphy cites and agrees with Ballaster's argument that 18th century women authors situated their own work within the male tradition and did not seek recognition for the influence of other female authors. However, Murphy argues that Austen makes obvious the connection to Brunton.

Here, Murphy makes a claim and supports it with a secondary source on the place of literature in English education. In doing so, Murphy is illuminating the basis of Austen's assumption of a certain body of literary knowledge.

Here, the author turns to a primary source, a letter by Jane Austen, to research her views of her readers. While authorial intention is often inscrutable, such a primary source can assist in evaluating Austen's notions of readership.

This is what we mean with the term 'sociological imagination'. Theory allows us to 'see' below the surface of society and to understand the invisible network of norms, values, structures, institutions and systems of inequality that shape individual choice and behavior. In this way, SNA should be fundamental to sociological methods.

The challenge of descriptive research for traditional social scientists is the change in how questions are asked. Questions should focus on describing something in a deep and informative way. Traditional social science relies more on predictive and inferential analysis. Can I predict what will happen if this variable changes in this way? Research questions identify the independent and dependent variables. SNA does not have IV and DV so questions are more about revealing what is going on underneath; i.e. how do the members of corporations know each other?

It is the blessing and the curse of SNA--it can do so much but it can also do too much. The analyst has to be clear in their question, defining what is a node and what is a link. It can get even trickier since nodes and links can also be reversed. Sometimes a node can be a link and a link can also be a node!

Excellent! Therefore SNA requires three points of data--Node A, Node B and the link between them. There is no dependent and independent variable. The means there is no inferential or predictive questions. Questions are more descriptive and comparative.

There are three points of data in SNA; node A, node B and the link between them. Traditional social science requires only two--independent variable and dependent variable.

Isn't this the foundation of the sociological imagination? Why is SNA not more central to sociological methods?

As I think about this, I am always a bit perplexed as to why SNA is not more foundational to Sociology. SNA reveals that which is very fabric of our society. Why is is not more utilized as a methodology? I suspect it has something to do with how hard it is to collect data.

These outcomes have been linked to friends and social contacts. Research asks how many friends or how often do you socialize? While this hints at the issue of networks, asking for lists or numbers does not produce network data. You have to find the links between people and between those people that people know.

Social networks are like 'air'; they surround us and we don't even see them. To me, this is what makes the methodology of SNA harder to grasp; how to access the data for 'air'? How to understand the discreet influence of 'air'? But once you see it, you can't unsee it! Very powerful!

What would be the links between these people? If you asked someone to list their friends, you will get lists which produce a star network. There needs to be a second round of questions involving friends of friends. Getting network data requires asking interrelated people.

This perspective of looking at individuals in categories is the foundation of statistics--two variables that are mutually exclusive and the goal is to see if they relate in any way. SNA is very different--all variables are related and dependent. That is why SNA is descriptive--can't do predictive without mutually exclusive variables.

Different methodology for research.

What this doesn't tell us is how many of the faculty/students were reporting experiences on the same projects.

Page 8

Jockers talking about the old approach in the 1990s to anecdotal evidence:

… in the 1990s, gathering literary evidence meant reading books, noting "things" (a phallic symbol here, a bibliographical reference there, a stylistic flourish, an allusion, and so on) and then interpreting: making sense and arguments out of those observations. Today, in the age of digital libraries and large-scale book-digitization projects, the nature of the "evidence" available to us has changed, radically. Which is not to say that we should no longer read books looking for, or noting, random "things," but rather to emphasize that massive digital corpora offer is unprecedented access to literally record an invite, even demand, a new type of evidence gathering and meaning making. The literary scholar of the 21st-century can no longer be content with anecdotal evidence, with random "things" gathered from a few, even "representative," text. We must strive to understand the things we find interesting in the context of everything else, including a massive possibly "uninteresting" text.

Pages 7 and 8

Jockers is talking here about Ian Watt’s method in Rise of the Novel

What are we to do with the other three to five thousand works of fiction published in the eighteenth century? What of the works that Watt did not observe and account for with his methodology, and how are we to now account for works not penned by Defoe, by Richardson, or by Fielding? Might other novelists tell a different story? Can we, in good conscience, even believe that Defoe, Richardson, and Fielding are representative writers? Watt’s sampling was not random; it was quite the opposite. But perhaps we only need to believe that these three (male) authors are representative of the trend towards "realism" that flourished in the nineteenth century. Accepting this premise makes Watts magnificent synthesis into no more than a self-fulfilling project, a project in which the books are stacked in advance. No matter what we think of the sample, we must question whether in fact realism really did flourish. Even before that, we really ought to define what it means "to flourish" in the first place. Flourishing certainly seems to be the sort of thing that could, and ought, to be measured. Watt had no yardstick against which to make such a measurement. He had only a few hundred texts that he had read. Today things are different. The larger literary record can no longer be ignored: it is here, and much of it is now accessible.

Page 217

Methods for organizing information in the humanities follow from their research practices. Humanists fo not rely on subject indexing to locate material to the extent that the social sciences or sciences do. They are more likely to be searching for new interpretations that are not easily described in advance; the journey through texts, libraries, and archives often is the research.

Page 213

Humanities scholarship is even more difficult to characterize than are the sciences and social sciences. Generally speaking, the humanities are more interpretative than data driven, but some humanists conduct qualitative studies using social sciences methods, and others employ quantitative methods. Digital humanities scholarship often reflects sophisticated computational expertise. Humanists value new interpretations, perspectives, and sources of data to examine age-old questions of art and culture.

Discussion of the methodological difficulties involved in close reading: i.e. there is no such thing as "just reading."

Culler, Jonathan. 2010. “The Closeness of Close Reading.” ADE Bulletin, 20–25. doi:10.1632/ade.149.20.

Discussion of the methodology of close reading: middleton, Peter. Distant Reading: Performance, Readership, and Consumption in Contemporary Poetry. Tuscaloosa: U of alabama P, 2005. Print.

Page 16

One benefit of traditional hermeneutical practices such as close reading is that the trained reader need not install anything, run any software, wrestle with settings, or wait for results. The experienced reader can just enjoy iteratively reading, thinking, and rereading. Similarly the reader of another person's interpretation, if the book being interpreted is at hand, can just pick it up, follow the references, and recapitulate the reading. To be as effective as close reading, analytical methods have to be significantly easier to apply and understand. They have to be like reading, or, better yet, a part of reading. Those invested in the use of digital analytics need to think differently about what is shown and what is hidden: the rhetorical presentation of analytics matters. Further, literary readers of interpretive works want to learn about the interpretation. Much of the literature in journals devoted to humanities computing suffers from being mostly about the computing; it is hard to find scholarship that is addressed to literary scholars and is based in computing practices.

Page 6

Computer-assisted research in the humanities, by contrast to the Cartesian story and traditional humanities practices, has almost always been collaborative. This is due to the variety of skills needed to implement digital humanities projects. It is also linked to the relationship between the practices of interpretation in the development of the tools of interpretation, be the tools for analyzing text or digital editions. Anyone who has used tools forged by another person is in collaboration, even if one isn't personally influencing the provider of the tools. The need to collaborate, though acknowledged in various ways, has been a professional hindrance, as anyone who submits a curriculum vitae for promotion listing nothing but co-authored papers knows.

Pages 6-7

Collaboration is not always good. It separates the interpreter/scholar from the designer/programmer who implements the scholarly methods. Willard McCarthy notes that the introduction of software "separated the conception of the problems (domain of the scholar) from the computational means of working them out (baliwick of the programmer) and so came at a significant cost.” As computing is introduced into research, it separates consumption, implementation, and interpretation in ways that can be overcome only through dialogue and collaboration across very different fields. Typically, humanities scholars know little about programming and software engineering, and programmers know little about humanities scholarship. Going it alone is an option only for the few who have time to master both. The rest of us and up depending on others.

Indeed, a major advantage, in terms of methodology. Bias is introduced in myriad ways, but that one way would likely have a much deeper effect than many others.

Really interesting idea: do an analysis of humanities researchers in general (DARIAH) and Holocaust researchers in specific (EHRI). One is very heterogeneous, the other very homogeneous (at least in terms of working conditions and, broadly speaking, data sources).

description of "arts and humanities researchers"

Benardou, Agiatis, Panos Constantopoulos, and Costis Dallas. 2013. “An Approach to Analyzing Working Practices of Research Communities in the Humanities.” International Journal of Humanities and Arts Computing 7 (1–2): 105–27. doi:10.3366/ijhac.2013.0084.

"it is wise to avoid generalizations and to concentrate instead on showing how interactions between coworkers, specifically the orchestration of information exchange and coauthorship, are grounded in local culture."

Hawhee: why do I buy the "profound," the "most profound" as a description of sensory knowledge?

Is it not the task of simplifying, that the research denies herself, when dreaming of showing the full complexity of a phenomenon to it audience?