Do you think they have covered limitations accurately? Comment also on their suggestions for future work.

Here, the researchers focus more on practical applications than on theory. Do you think their findings are strong enough to justify these practical conclusions? Why or why not?

Do you agree with how the researchers evaluated their findings in this section and drew theoretical conclusion? Why or why not?

Same question as above with logistic regression. Often, knowledge of a simpler technique will give you insight into the basic workings of a more complex one.

This is, yes, a more advanced technique than we have time to study this semester. Can you make any sense out of it using what you know about regression analysis from chapter 12?

Comments on this table?? Would anyone care to add information about Cohen's d? The "Effect Size" entry apparently refers to the Pearson's correlation correlation coefficient (see text below table referring to t-test results) between gender and the 6 DVs in the rows.

Pearson's correlation coefficient (r) in a common format used when including it in the text.

Language like this is often used if researchers don't quite meet certain standards of data analysis ("exploratory study").

We're not doing factor analysis in this introductory course, but it's a good idea to get some basic familiarity with it so that it isn't mysterious to you. EFA means "exploratory factor analysis." Here's a link to a good, basic explanation of EFA using SPSS: https://stats.idre.ucla.edu/spss/seminars/efa-spss/. Probably too long for an end-of-the semester read, but quite practical in its approach.

Do these research questions give you any ideas about what kinds of data analysis techniques the researchers will use later? Should research questions imply analysis techniques?

Is there any more information you'd like to have included in this section? Do you think there's enough here for researchers to try to replicate this study? As always, please explain!

Hi, all. I am not making any comments on the first part of this article - research questions, theory, etc. Instead, I'm heading straight to the methods and results sections to concentrate on those. Feel free to make comments on the front end though if you would like.

Do tables 4 and 5 make any sense now that you’ve studied independent samples t-tests and one-way ANOVA? Would you construct these tables differently or change anything about the written interpretations?

Table 3 - Do you think this table is useful? Why or why not? If you have a different way to arrange the results or a different approach to the analysis, please summarize it here. We'll stop right now with this table and pick up again later with tables 4 and 5.

Even if this is the case with regard to mean scores, do you think there might be some differences in the distribution from 1-5 using percentages? How could you evaluate the data to reveal some of these differences?

What do you think of these questions from a methodological viewpoint? From what I can tell on a quick examination, some in table 1 (real life) are the same as in table 2 (SNS), but others are different. Can you discern any pattern or reason for these similarities and differences? Also, is there any rationale for the arrangement of the questions in tables 1 and 2?

How do the researchers assign these rankings? They are obviously quite uniform. Do you think the researchers needed to include all of the information in tables 1 and 2? And on another matter, these are ordinal level variables. Do you think calculating mean scores was the best way to do this analysis? What about doing something else? What else exactly?

This is a measure of instrument reliability with regard to internal consistency. If I'm teaching 602 when you take it, we will do some of this in SPSS. Also, I'm curious about the language from George and Mallery about the "acceptable" and "good" levels. I think it would be interesting if someone did some searches to see if other language is used by other researchers.

What do you think the actual responses to the 5-point Likert scale looked like on the questionnaire?

What are some of the procedures the researchers could have used to draw the sample randomly? From what they say, it seems as if all of the students chosen randomly agreed to participate in the study. Do you think this might be the case?

What do you think this kind of language means? Here, I mean what do you think the difference is between direct prediction and indirect prediction?

To me, this kind of language screams gap, gap - gap in the literature that we have to fill! How else do the researchers use the literature to support undertaking their study?

Do you think the researchers give a good definition of SNS? Why or why not? If not, can you think of a better one? (It would be interesting to do a search on definitions!)

The abstract gives an overview of the study. How would you express the research question (or questions) based only on the abstract?

We'll tackle this article in two steps. First, we'll discuss the research question, methodology, and tables 1-3 of the data analysis. You currently have sufficient background in statistics to discuss those tables. Later on, as we cover more sophisticated forms of analysis, we'll move on to tables 4 and 5.

Do you think the researchers accurately characterized the exact text of these questions?

Do the researchers define these terms anywhere? What do you think them mean in the context of the article?

Do you think the researchers' depiction of the results in Table 1 is accurate? Why or why not?

Here, the researchers are referring to their distinction between respondents living in the South region and those living in the "Non-South" region. The GSS classifies respondents' geographic location into one of nine regions, but do not have variables for states or local levels. Why do you think this is?

Each column in this table would be a valid percent column in an SPSS frequency distribution table. Do you understand everything reported in this table? Would you change anything about this table or include more information in it?

Also, maybe somebody can look up these exact variable names on the GSS Data Explorer website: https://gssdataexplorer.norc.org/. When were these questions asked?

Does this speak to criteria for causality? Why or why not?

What do you think are the strengths and weaknesses of this study? What suggestions would you make for further research?

Same questions about all tables in the article. Do you understand them? Would you change anything about them? Are the summaries of the information in the tables correct and adequate?

Since we will be using the most recent GSS for the SPSS assignments, can somebody look up a better (but short) description? Broadly speaking, what sort of sampling technique was used (e.g., simple random, stratified, cluster)? What does "non-institutionalized" mean? What is the age restriction?

Now, usually you'll see an abstract before the actual beginning of the article, but there isn't one here. See if you can identify one or more research questions near the beginning.

Welcome to our first Hypothes.is annotation/discussion! The goal of this assignment is to give you some experience with critiquing the methods and statistics sections of research articles. I'm not quite sure of this right now, but I think almost everything I post will be based on secondary data analysis of GSS surveys. In your SPSS assignments, you will become more familiar with the basic features of these data files. Also, if you are curious, you can go to the GSS website and download datasets from more recent years to look at trends. For all of the articles I post, I will make introductory comments and contribute to the ongoing discussions. Remember, it's okay to get things wrong - or not quite right! The grade is based more on participation than on application of academic knowledge.

This is an interesting sentence. It acknowledges sample limitations, but alleges "provocative evidence." Does the researcher present suggestions for future research in the conclusions?

This is what Nicol and Pexman call a "word table."

Verbatim quotes highlight what the researcher considers to be important points. This is very common in qual research reports based on interviews or focus groups or some other form of verbal data.

Here, the language is "provides some preliminary support," rather than a discussion of statistical significance or strength of relationships.

I wish the author had discussed the IRB board's requirements!

By the way, VCU has this program and at least one other qual analysis software program on the App2Go server for student research use.

This is a common justification for collecting data on a limited number of cases in qual studies. What is "data saturation," who makes that judgment, and is it an adequate standard for stopping data collection?

This is a more detailed explanation of research methods than you will often find in qual articles. For example, the researcher details a definition of "profoundly visually impaired" and pretty clearly discusses recruitment techniques.

A gap in the literature. Looks to be another research question, but are the questions organized systematically, as they would be in a quant article?

A listing of some research questions. If this were a quant article, we'd hold the researcher to providing some kind of answer for each of these questions. Should we do that in a qual article? Does the author follow through with answering every one of these?

In a quant article, we also would hold the researcher to a clear definition of terms. For example, a quant researcher might be brought to task in the first research question for failing to specify "effects" on what or for having a causal research question inadequately addressed by the analysis techniques. Would we do this here? If not, then what standards would we use for determining whether the researcher has adequately addressed the questions?

Situating race in a context of visual perception. I won't go into theoretical detail with my annotations, but feel free to do so!

Qualitative research articles often feature less precise and more exploratory research questions than quantitative ones. Henceforth, I will use abbreviations for qualitative (qual) and quantititative (quant).

Here, the researchers say that they excluded any case that had missing data on any of the variables they examined. Why do you think think they used this particular solution to the problem of missing data?

This is the only paragraph I can find that is devoted to study limitations. Do you think the researchers should have expanded this discussion and/or put it in the Discussion section?

I don't recall seeing an exclamation point here instead of p<.10, which is what it means. Has anyone else seen this in their literature travels?

Since all of these variables are dichotomous, the mean scores represent the proportion of the sample that has a score of 1. And, of course, multiplying proportions by 100 yields percentages. So, for example, 29% of the sample has less than a high school education. Though the table doesn't specifically say so, the fact that they note significance levels refers to whether the difference between sanctioned and nonsanctioned families is statistically significant or not (independent samples t-test).

Here's a good example of a conclusion to a literature review. The first sentence identifies a gap in the literature and the second explains what this study does to address that gap.

It's relatively rare for a sophisticated quantitative article like this to include simple charts. I think it helps orient the readers. However, in this case, I'm not sure why they used histograms rather than bar charts. Thoughts?

This is a helpful table that Nicol and Pexman would term a "Word Table."

The problem here is that some items have three values (T, F, DK) and some have two values (correct, incorrect). If the researcher just sums questions of both types into an index, the items with 3 values will count more heavily toward the total score than those with 2 values. Z-score standardization of all items is a way to correct this. It's one of the options I give you to use on the upcoming index construction exercise.

Here, it seems as if the author will really be testing whether or not individuals avoid these record-keeping institutions. Surveillance now seems to mean something different, as in "pays close attention to a person." The next annotation by Ryan supports my comment here about the paper researching avoidance, rather than actual surveillance - so do the later data tables.

I'm wondering if there's a better way to say the meaning of the zero code. I'm trying to think about this logically. I think there are two alternatives with two possible answers. Did R get medical care - Yes, No Was medical care necessary - Yes, No So, let's look at the possible combinations:

1. Got=Yes, Needed=Yes
2. Got=Yes, Needed=No
3. Got=No, Needed=Yes
4. Got=No, Needed=No Please correct me if I'm wrong, but I think 3=did not obtain necessary medical care 1, 2, 4 = did not report no obtaining necessary care Is this any better? [1=did not get medical care when needed 0=did not get medical care when not needed or got care whether or not needed] Or maybe "0=might have gotten medical care, even if not needed"?? I think I give up.

I wonder why the researchers did not explore the possibility that these 3 sampling techniques might have resulted in differing demographics or food choices.

Can you think of other research possibilities?

Do you think these have been covered adequately?

By the way, this table displays the results of the multiple comparison test (MCT) that Austin and I were discussing earlier. I think the numbers in parentheses are standard deviations, but they don't say what these are within the table. Actually, they confirm this in the text, but why not note it in the table also? Table 2 displays the ANOVA results but not the F-value or exact p-values.

I think this statement is vague. How was it used? The rest of this recruitment description is pretty precise. The ideal is to write your methods section as thoroughly and precisely as possible, so that another researcher could replicate your work. Unfortunately, that is rarely done, which I think is very problematic.

Thanks to Giny and Ryan for carrying through to the end. I kind of just shook my head at some point and jumped over a lot of stuff and went straight to the end. I am underwhelmed.

There's evidence lurking behind the choice of probability-based sampling to represent larger populations of interest. This is not an assumption. As Ryan points out, survey researchers do not assume relative stability of results over time. If they did this, clearly they would not undertake any new surveys!

Okay, so I feel better now that the author has stopped being so critical of surveys. In my experience, social scientists are appropriately aware of the limitations of surveys. Perhaps in a sense the author is making a "straw person" argument, in that surveys cannot possibly be designed to give a perfectly accurate measure of an objective psychic reality. Perhaps the author's main argument though is whether "public opinion" actually exists as a measurable construct in reality. I'm not a theoretician and can't critically the arguments knowledgeably, but I can't discern any practical effect of all of this on sociological research. Perhaps one or more of you can educate me properly!

Also, what do you think about the structure of these tables? For example, in an academic paper, you should see the total number of cases on which the results in Table 1 are based. Is this a problem?

Here's one thing I'll point out though. This actually is not a probability sample in the sense that you learned in 601. Participants are solicited, not chosen randomly using a sampling frame. Then the sample is weighted to reflect the composition of the U.S. adult population. Here's the Harris Poll online panel welcome page: Link Do some more exploration of this methodology. What do you think of it?

Here's the fun part, finally! I'd love to see you all read this section carefully and make astute comments! After all, it is a research methods course.

There's relatively little attention devoted in the body of the report to the methodology of the study. The blue box describing the sample a couple of pages later is the most methodological information in the body. Appendix 2 provides a detailed discussion of the methodology, comparable to or maybe even more extensive than we might see in an academic report. It's assumed that most readers would not be interested in that, which is definitely not true of an academic report.

Factor analysis is a more complex quantitative data analysis technique than we covered in 508. Perhaps one of you could delve into it a bit and determine whether you think it's a reasonable approach in this study. In order to understand quantitative articles, it's important to develop some general familiarity with the purpose and general approach of analysis techniques, many of which will be well beyond the coverage of an introductory course. I don't ask you to actually use any of these more complex techniques in your DAPE work or project, but I do think that master's level students should develop the facility to learn some general information about quantitative analysis on their own.

Is there anything here that you would call a theory? Explain your contention.

This introductory section is not labeled as part of the "literature review," but is it? In what ways are the two sections similar and different? Do they have different purposes, structures, or what?

Here's a table displaying techniques that were covered in 508 (p. 1334). Is there anything about the table that you don't understand? If so, perhaps one of your classmates can help.

Do you think the discussion adequately characterizes the results? Have the researchers made any broad, unjustifiable statements? Can you think of any other limitations than those that have been acknowledged?

Would someone please explain what this means? Why do you think researchers rely so much on this criterion?

This section of articles usually offers plenty of material for critiques.

Welcome to our second article! My main purpose is presenting and having you discuss these research articles is to acquaint you with the structure and functions of typical research articles in Sociology. Also, it can function as a critiquing exercise, considering what's good and not so good about research articles. As you read each section, comment on what you think the author(s) are trying to accomplish and whether or not they succeed.

Would anyone like to take a stab at explaining what a linear regression model does? (Actually, this is a linear multiple regression model.) Look up some information if you need to.

The researchers do not present any bivariate analyses. Do you think they should have? Explain.

Do the researchers give you sufficient information, either in the table or text, to interpret this mean score? Be sure to explain this and not just say "yes" or "no"!

To what extent do you think these statements are good measures of fear of criminal victimization at school?

Considering this paragraph, do you think this study is inductive or deductive in its approach?

What do you think the researchers are trying to accomplish here?

How do you think this variable was handled in the data analysis? What would you do in SPSS to set it up? Three variables in this table, Black, Male, and Poor are measured in a similar way.

What do you think of how this variable is measured?

What do you think of how these variables are measured?

This approach is called "active parental consent." Any idea what the opposite approach is called and how it's done?

This "void in the literature" approach is a common justification for doing research in a particular area. Maybe a little less dramatically, we can call it the "gap in the literature" approach. When you're doing a lit review, it's a good idea to pay attention to topics that have received little or no attention.

I agree with Ryan that this section is important, but I see it more as the beginnings of thinking about theory/theoretical framework than the section discussing recent adolescent crime trends. However, I definitely agree that the authors are slowly building a case for their overall approach.

Academic journals differ in their citation and references styles. This particular journal (Sociological Spectrum) requires use of American Sociological Association 5th edition format. However, I've asked you to use APA format in this class, since it's very common in social sciences. Here's a link that summarizes APA style.

The "Discussion" or "Conclusions" section is supposed to pull everything together. The authors are supposed to reflect on what the results mean for their research questions and hypotheses (if any), the importance of their results, the limitations of the study, and the implications for future research. Is that what's happening here?

This is really a "Results" or "Findings" section. Often, the details of the statistical approaches are discussed in the Methods section instead of, as here, the first part of a Results section. It will be difficult for you to evaluate whether or not the statistical methods of academic articles are appropriate or not. I have two suggestions:

1. If possible, use what you already know about statistics to help you understand the analysis. Here, the authors use a multivariate linear regression model with more than one independent variable. Use what you know about bivariate linear regression to get a basic idea about what's happening here.
2. Do some internet research to get a basic overview about what multivariate linear regression is all about. You don't have to get down to the finest details to have some understanding of the methodology and whether it fits these particular analysis circumstances.

Here's our first article for annotation! The idea of this whole series of articles is to give you a structured opportunity to give very close attention to the organization, structure, and content of published academic articles. As a result, I hope the quality of your own academic writing will improve. I will make observations on these articles as well.

The goal of a Method (or Methods) section is to describe the research methods in sufficient detail for another researcher to be able to do the same or a similar study, assuming access to resources. Do you think this goal is met? Is there anything missing?

This section, going all the way down to the Methods section combines introduction/background, literature review, and theory/conceptual framework components. Can you distinguish one from the other by considering the purposes the authors have as they go along?

Here's an example of an annotation.