140 Matching Annotations
  1. Sep 2023
  2. Feb 2023
    1. raining”

      Introduction to the role of "training data" in machine learning: https://www.youtube.com/watch?v=x2mRoFNm22g

    2. needs

      Needs-aware AI has been proposed: https://arxiv.org/abs/2202.04977 (short) and https://arxiv.org/abs/2203.03715 (longer)

    3. designed with a set of values,

      Sometimes we may appreciate these, such as when the designers "values" align with our "values". Other times we may not appreciate the "values" embedded into the design when they contrast with our "values".

    4. fairness

      Here is a good video introduction to the tradeoffs of fairness and accuracy in machine learning: https://youtu.be/p5yY2MyTJXA?t=230

    5. unfair

      This is a good article on what does it mean to be "fair":

      What Is Fairness? Philosophical Considerations and Implications For FairML https://arxiv.org/abs/2205.09622

    6. environmental

      Here is a review on the latest research about "Green AI": https://arxiv.org/pdf/2301.11047.pdf

    7. Rebecca Eynon
    8. Nabeel Gillani
    9. importance

      Looking forward into the future, this video provides some useful considerations on how AI might change our lives: https://www.youtube.com/watch?v=RzkD_rTEBYs

    10. generate

      These are often "Generative AI".

    11. Machine learning models are designed to identify and exploit correlations
    12. grasping a rich understanding of causal processes in settingsas complex as education usually requires much more than technical solutions

      This is a critical point and expanding here by Judea Pearl: https://www.youtube.com/watch?v=CsMV5o3hotY

    13. Linear regression

      If you are not familiar with Linear Regression, here is a quick introduction to the concept: https://www.youtube.com/watch?v=iIUq0SqBSH0

    14. Machine learning

      If you are not familiar with machine learning (ML) here is a very basic introduction: https://www.youtube.com/watch?v=f_uwKZIAeM0

    15. frequentist

      If you are not familiar with this term, this video may help: https://youtu.be/dejy4PPCFHY?t=120

    16. uman-centered AI

      Human-Centered AI (HCAI) is a sub-field of AI, here is an introduction by a colleague at the Univ. of Maryland: https://www.youtube.com/watch?v=bgJZ97q3Q9w

    17. Georgia State University

      Here is more on the GSU "proactive advising" program: https://theuia.org/blog/post/the-national-institute-for-student-success

      And a video interview: https://www.youtube.com/watch?v=FdR8sI0D5i8

    18. ITS

      Here is an example of ITS being used to Georgia Tech: https://www.youtube.com/watch?v=C79x_tK1mg0

    19. biased

      Though using the term "bias" here, this is not referring to "human bias" (e.g., prejudice) but rather statistical/computational bias. Here is a good video on the bias-variance tradeoff (statistical bias) in machine learning, then later in the article they get to "human bias", both are important: https://www.youtube.com/watch?v=EuBBz3bI-aA

    20. biases

      Here now they talking about "human bias", which is different form the technical bias-variance tradeoff discussed before, though the two do relate to each other at times. Here is more information on "human bias" in AI: https://www.nist.gov/news-events/news/2022/03/theres-more-ai-bias-biased-data-nist-report-highlights

    21. but their inner workings are usually nottransparent, making them difficult to interpret.

      See the video above to understand why this is. Most tools for explaining AI outputs (a field known as Explainable AI or XAI) are thus post-hoc (ie., done after to determine what factors had the most influence on the output). Even then, interpreting the results can be complicated.

    22. “GPT-3” language model

      More recently (Dec. 1, 2022) the company OpenAI release ChatGPT (or GPT 3.5). You can watch this video of the workshop we did at GW on ChatGPT (Jan. 18, 2023):


    23. neural networks

      If you are unfamiliar with neural networks, this video explains them well:


    24. Bayesian

      If you are unfamiliar with this term, watch this video:


    25. demystifying AI

      This is the definition I use:

      "Artificial intelligence is the science and engineering of making computers behave in ways that, until recently, we thought required human intelligence. "

      Andrew Moore, former Dean of the School of Computer Science at Carnegie Mellon University (in Forbes, 2017)

  3. Jan 2021
    1. You’ll need to break knowledge down into its discrete components so that you can build those pieces back up as prompts for retrieval practice.

      For carefully written manuscripts, authors will have done this in the writing process already. Instructional designers, for instance, routinely do this and it becomes habit.

  4. Jul 2019
    1. .616*

      Correlation between Pre-Class Engagement and In-Class Engagement. In other words, those who were engaged Pre-Class were also typically engaged In-Class. For every 1 point increase in "engaged Pre-Class" there was a 0.616 increase in "engaged In-Class".

      Here is a short review on interpreting correlations (sorry about the ads on the page): https://www.dummies.com/education/math/statistics/how-to-interpret-a-correlation-coefficient-r/

    2. M (SD)

      Means (M) and Standard Deviations (SD)

    3. p<.01

      Reminder on interpreting the p-value:


    4. 70

      Though statistically significant, a correlation of 0.27 is fairly low and may not represent any practical significance.

    5. r

      r = correlation

      Correlations go between -1 and 1, with 0 being no correlation.

    6. Table 2:

      Here is a good article describing how to read a regression table: https://www.statology.org/how-to-read-and-interpret-a-regression-table/

    7. Underlying assumptions of the three statistical analyses were verified with Shapiro-Wilk’s test, Levene’s test, the VIF and tolerance values, Dubin-Watson statistics, the scatter plot of the values of the residuals against the values of the outcomes predicted, and the histogram and normal probability, from which the assumptions of multi-collinearity, linearity, homoscedasticity, and independent residuals were confirmed

      These are just multiple ways that researchers can confirm that their data meets the underlying assumptions required in order to do the regressions. Here is a good overview of the basic assumptions: https://pareonline.net/getvn.asp?n=2&v=8

    8. χ2

      Chi Square

    9. goodness-of-fit indices

      Here is a good video describing Pearson Chi Square, which is one of the most common "goodness-of-fit indices" and it thereby explains the basic concept:


    10. maximum likelihood estimation.

      Video on MLE:


      And for those who want to get a little more technical:


    11. To explore the path from influencing factors to FL outcomes

      Refer back to Figure 1 to visualize the paths (the arrows between the boxes). This is a fairly simple path model and it is easy to interpret, but they can get more complicated with more arrows going in different directions between variables.

    12. effect sizes

      Video on effective sizes (including Cohen's recommendations):


    13. hierarchical multiple regression

      Here is a video overview of multiple regression. It builds on the previous video on simple linear regression (start with this video if regression is a new topic for you):

      Multiple Regression: https://www.youtube.com/watch?v=dQNpSa-bq4M

      Linear Regression: https://www.youtube.com/watch?v=ZkjP5RJLQF4

    14. analyses of variance (ANOVA
    15. bivariate correlation
    16. participants’ perceived growth

      The extent to which we believe that students can make accurate estimates about their growth from the beginning of the semester to the end of the semester should be considered as we later look at the data on this variable.

    17. and

      Again, this would better as two questions.

    18. lectures and materials

      Typically researchers will not use "and" in a question since it indicates that you are asking two questions. For example, if a participant here used the pre-classes lectures but not the other materials, should they mark yes or no. It would be better asked as two questions (one for materials and one for lectures).

    19. selectively adopted items from a ques-tionnaire developed by Hofer and Pintrich (1997) and translated and validated by Cho (2010)

      While it is somewhat common for researchers to select only a subset of questions from a validated instrument, it should be noted that the instrument was only valid in the form that was validated. Variation on the validated instrument may or may not be valid, and a separate study should be done to validate the subset of items to see if they are still valid (and reliable). In this study they do provide the "internal consistency" scores for the items as used, but this is more about reliability (i.e., does it consistently measure what you want to measure) and not as much about validity (i.e., is it accurately measuring the desired construct).

    20. elected and modified item

      Note again, they modified an instrument.

    21. Cronbach’s α =

      Cronbach's alpha = measure of internal consistency of items

    22. three sub-constructs

      It is not clear why this is not labeled with the type of coefficient (number) that it is. It is likely the Cronbach's alpha (the most common measure for internal consistency) but later on the page they describe another coefficient as the Cronbach's alpha, thus we are left to wonder if these are or are not similar.

    23. 0

      This looks to be an editing error since the value should between 0 and 1 (e.g., 0.7).

    24. internal consistency reliabilities

      Video on internal consistency:


    25. perceived learning outcomes

      Note that this is not their actual learning outcome, but what they thought that they would get.

    26. At the end of the semester

      All data was collected at the end of the semester from the students. This is not necessarily a problem, but when reading the results we should keep in mind that this captures where they are at the end of the semester and not at the beginning (for example, questions about their engagement would be attached to the engagement at the end of the semester and not at the beginning or middle of the semester).

    27. 0 to 60 minutes to learn.

      It appears that this is an estimate based on instructor experience rather than data actually collected from the students. Given that the data on student use would be in Blackboard (the LMS) it would have been good for them to include the actual time students spent pre-class engaging with the materials.

    28. two 15-week courses

      A physics and a chemistry class were used in the study, both used a FL approach. There was no comparison group of students (ideally randomly assigned) taking the same courses but using a FL approach.

    29. research questions

      Research questions are an alternative to making hypotheses, though based on the research literature generally researchers should make testable hypotheses. Typically they will leave it at research questions if (a) there is not enough supporting past research to make a good estimate of what is expected, or (b) they haven't done enough background research and therefore can't make a good estimate of what is expected.

    30. Figure 1:

      Figures are often quite useful for understanding what the researcher is doing or proposing, so take time to study them.

    31. Even though learner engagement typically comprises cognitive engagement, behavioral engagement, and affective engagement (Archambault, Janosz, Fallu, & Pagani, 2009; Fredricks, Blumenfeld, Friedel, & Paris, 2005; Fredricks & McColskey, 2012), in a context such as FL, engagement should consider the student’s initial commitment to an online pre-class learning mode (hereafter, pre-class engagement) and subsequent commitment to a F2F in-class learning mode (hereafter, in-class engagement).

      This is an important statement since the authors describe how they are going to define "engagement" within the context of this study.

    32. pedagogy

      Is FL a pedagogy or tool? You can decide:


    33. epistemological

      Here is a video introducing what is "epistemology":


    34. identified diverse out-comes associated with FL course

      Again, though technically accurate this may be a little too strong of a statement given the mixed results of FL.

    35. Studies have documented

      As mentioned in Molnar (2017), these finding have actually been quite inconsistent. So while it is true that some studies have documented improvements, others have not.

    36. (Molnar, 2017

      This is a good example of why it is important to look at the references linked to citations. Molnar's study was not a broad examination of the "numerous reports of improved academic performance and enhanced learning". Rather Molnar actually looked mostly at student perceptions of FL learning. Molnar did include some information on grades, but only for part of the sample and with limited control conditions. So while Molnar's article may generally support that some research has shown positive impacts, it balances that with "Performance measures between traditional and flipped classrooms have also generated inconsistent findings." In the end, after looking at Molnar, I do not think that I agree with the author that Molnar's article is a citation for this statement (not that the statement may not be true, but this conclusion was not the results of Molnar's research).

    37. Practitioner Notes

      This section is a standard element for the British Journal of Educational Technology, but it is not common to most research journals. Since the statements do not include citations, you will want to cross check them with statements in the article to determine if you agree (through the citations) that the statements are warranted.

    1. partialη2

      n squares (or eta squared) is a statistical tool for examining the effect size of a result. The effect size is an indicator to readers distinguish between results that are statistically significant and those that are practically significant, though in the end the practical significance (or utility) is left to the interpretation of the reader. For example, a difference with a substantial effect size may have practical significance for teachers in urban schools, but at the same time have little practical significance to those in rural schools, based on other characteristics of the study (such a participant profiles, the technologies used, capacity of internet access, etc.).

      This video gives a good explanation. https://www.youtube.com/watch?v=RkbmA6WszTo

    2. Chiu

      Depending on the topic and discipline, self-citation (i.e., referencing your own work) can be acceptable. But readers would want to note if the author(s) reference primarily their own work since this could be an indicator of potential bias in the article. Though in some cases, very few researchers are working on a given topic and there are not many others to cite... thus there is no clear "rule" on how much self-citation is acceptable.

    3. References

      Reference sections are very important to science. Not only do they allow readers the opportunity to review (and interpret for themselves) the research being cited by the authors, but they also give readers a quick way to discover what influential articles are guiding the researchers. Many seasoned readers of research will start by skimming the References before going back to read the article -- as a preview of coming attractions.

    4. University of Hong Kong

      Researchers should also acknowledge who is paying for their research. It is not that research paid for by group's with special interest is "bad" research, but readers should be aware of any potential connections (similar to the conflict of interest statement above).

    5. conflict of interest

      In this section the authors should indicate if there are conflicts that might influence how people interpret their research (for example, if research on lung cancer is being done by researchers who are owners of a vape company).

    6. limitations

      Researcher will commonly identify limitations to their research and these are important for readers since they may influence how you interpret the findings and implications.

    7. motivate all teachers to integrate mobile devices intotheir teaching practice

      This statement gets back to why it would be important for the researchers to monitor and measure the "use" of the mobile technologies of individual teachers in this research.

    8. **

      Be sure to note the legend below the table for the meaning of , , and . These are rather routine and typical, but sometimes authors will use different p-values.

    9. significant

      More accurately "...no statistically significant difference..."

    10. p= .01

      Reminder video on how to interpret the p-value


    11. F(1, 28) = 7.15

      This video does a good job of describing how ANOVA calculations are made, and how the F test is calculated (including what the 1 and 28 would represent from this example).


    12. before (M= 3.03, SD = 0.74) and after(M= 2.74, SD = 0.59

      ANOVA are statistical tests for differences in variation between groups, or in this case pre-intervention measures and post-intervention measures. In other words, rather than just comparing the mean difference (3.03 vs 2.74), the ANOVA uses the variation (from the mean) for individual scores.

    13. 0.70

      With the posttest, we see that the math and science teachers are more in agreement (i.e., less variation and a lower standard deviation) than they were at the time of the pretest (i.e., dropping from .98 to .70).

    14. 0.98

      The "perceived ease of use" has a greater standard deviation than most of the other scores, and this illustrates that for math and science teachers there was greater variation in how easy they found the mobile devices to use (i.e., more spread in their individual scores).

    15. SD

      SD = Standard Deviation

      As a reminder, here is a short introduction to SD.


    16. adoption

      As with "use" above, "adoption" here is not defined and no data is provided on actual teacher use. Since there is no control group (i.e., without mobile devices) we don't really know with that data if levels of adoption influenced levels of anxiety, or if some other variables were more influential on levels of anxiety.

    17. used

      The "use" of the mobile devices is essential to this study and yet it is not "operationally defined". In other words, you want the researcher to define how they operationalize the key variables in their study. In this case, how will know if teachers actually used the mobile devices? How will they measure this? How will discern differences? For example, one teacher may use the devices on a daily basis and another may use them once a week -- are these measured and considered equivalent (i.e., do both cases fall into the definition that the teachers "used" the mobile devices)? This would be important to our interpretation of the research findings.

    18. teachers received training workshops

      It would also be useful for the researchers to include what was taught in these workshops, number of hours of training, and participation rates among teachers. Since the study is quasi-experimental (i.e., no control group) then as readers we would benefit from additional details on what interventions the teachers participated in during the study.

    19. got consent.

      It is not clear if the research got consent of the principal and the teachers, or just the principal. Typically, all participants have to give consent to participate in a study.

      You can watch this for more information on why informed consent is important to ethical research: https://www.youtube.com/watch?v=BXQHDCWSt-Q

    20. modified

      It is important to note if the researchers made any changes to the instruments that they are using in the study. It is common for researchers to make minimal changes in order for the questions to make sense to the participants, but it is critical that these do not go so far as to change the validity or reliability of the instrument. Unfortunately, researchers do often go to far and in this case they used a mix of instruments together. This is not necessarily "wrong", but as the reader we do have to question if this has influenced the validity and reliability of the original instruments -- thus compromising the results of the current study. There is typically not clear answer to this question, as in the current paper. But as the reader you should be aware of the potential problems that may stem from altering an instrument from its original state (i.e., the state was tested for validity and reliability).

      Here is an overview on validity and reliability of instruments: https://www.youtube.com/watch?v=O4FvB-W4Siw

    21. Likert-type questions

      Likert-type scales are those that you are commonly used to completing where 1 might indicate that you disagree and a 5 might indicate that you disagree. There is an interesting history to these scales and you can read more about them at on Wikipedia - https://en.wikipedia.org/wiki/Likert_scale

    22. n= 29

      "n" refers to the sample size. In this case there were 29 language and humanities teachers among the 62 total teachers in the study.

    23. re- and post-questionnaires

      Pre - Post designs are common in research, with the Pre being a measure taken before the intervention and Post being the application of the measure after the intervention. For control group studies, the control group would also have the Pre and Post measures but without an intervention in-between.

    24. 62

      Sample sizes are typically important since they relate to the "power" of a quantitative research study. Here is a quick overview.


    25. Hong Kong.

      Be sure to note demographics of participants in the research. Here, there location (i.e., Hong Kong) might be influential in how you interpret, and potentially apply, the results. Other demographics, such as age, gender, etc. may also be worth noting.

    26. Hypothesis 1

      Studies usually present the hypothesis in the "alternative hypothesis" format, rather the "null hypothesis" format, even though the later is the one that is actually tested. Here is a review of the two types.


      If you don't recall why scientists use the null hypothesis, here is a review.


    27. information,communications and technology (ICT)

      Wiki description of ICTs in Education, with lots of example use cases.


    28. Evans,2008

      It is important to look at the cited references for context. For example, you will note the Evan's research in 2008 looked at podcast use in higher education. Thus, when the researcher here is discussing "using devices in classrooms" the evidence they are applying is related to higher education, and not K12 education -- this may, or may not, influence your interpretation.

    29. quasi-experimental desig

      Introductory video to quasi-experimental research designs (i.e., studies without control groups).


    30. epeatedmeasure

      Short video introduction to Repeated Measures ANOVA


  5. Jun 2019
    1. ANOVA

      ANOVA = Analysis of Variance

      ANOVA is a statistical technique used to compare the variations found among two or more groups.

      Resources: Wikipedia

    3. DOI:

      A DOI is an unique Digital Object Identified. They get assigned to articles so that they can be tracked and easily located.

    4. a

      ORCID is a non-profit organization that offers unique identifiers for researchers to automatically link together their contributions.

  6. May 2019
  7. Apr 2019
    1. β = -.035(median 3partial 휂2 = .001, median n = 62,297, median standard error = .004, see Figure 1)


      R Jupyter Notebook with code for this analysis: https://rnotebook.io/anon/fa877c0366dc965d/notebooks/DRAFT%20OF%20CONCEPT.ipynb

      Or R Jupyter Notebook with Binder: https://github.com/matthewfeickert/R-in-Jupyter-with-Binder

    2. NHST

      Null Hypothesis Significance Testing (NHST)

    3. screen use
    4. grounded in SCA
    5. ng SpecificationCurve
    6. garden of forking
    7. Andrew K. Przybylski
    8. Supplementary Table 1
    9. Supplementary Table 3
    10. dolescent well-being

      Amy discussing measures of adolescent well-being:


    11. Specification Curve Analysis(SCA)27
    12. Millennium Cohort Study

      Millenium Cohort Study Variable Guide:


    13. Amy Orben
    1. : https://www.youtube.com/watch?v=tH7bVlGNjWU
    2. methods and best practices applicable to this approach
    3. solar maximum
    4. formal citizen science projectAurorasaurus
    5. conjugate andcoincident fly-through by a low-altitude spacecraft, such as Swarm, wasneeded to determine the in situ nature of STEVE
    6. significantly equatorward of the auroral ovalduring enhanced activity
    7. aurora
    8. Elizabeth A. MacDonald
    9. itizen science project Aurorasaurus
  8. Sep 2018
    1. making poor decisions

      Yes, reduce poor decisions -- but that isn't the same as guaranteeing good decisions.

      We must still realize that we probably can't incorporate all of the perspectives, so some will get left out.

    2. ach discipline’s perspective

      Do we have to view through each discipline? Or is it better for us (as interdisciplinary researchers) to question, listen, interpret, and communicate what the experts in each discipline see when they view it through their lens. Me looking through the lens of a biologist isn't of much good, but I can ask questions and listen, and then try to give interpretations.

    3. adequacy, not mastery

      This is key -- especially as the complexity of the problems is realized.

    4. they provide only partial understandings of the subject under study.

      Though, getting a "complete" understanding is next to impossible with many of the complex challenges we want to investigate. Even interdisciplinary will remain partial since it can't possibly go as deep into the all of the specifics for each sub-discipline.

    5. past experiences

      Influential professors, courses, readings? Parents?

    6. Internally, everything people experience and learn is “colored” by their epistemic position

      Can we really change our position, or do we just become more aware of the positions of others?

    7. phenomenon

      What are current examples of research "elephants"? AI? Poverty?

    8. partial understandings

      "partial understanding" seems key to this. It is not that any perspective embodies all the necessary dimensions, but that each is missing something. For instance, the sociologist will be missing somethings that the lawyer might pick up.

    9. conflict-ing insights

      Hopefully we can come up with some examples to discuss in class.

    10. gender distinctions

      This is an odd statement, esp. since the author gives no context for it.

    11. mere opinion or personal preference

      Which may lead us to discount the knowledge generated in other fields. Or even to downgrade the knowledge of our fields.

    12. epistemic

      Related to the word Epistemology: https://en.wikipedia.org/wiki/Epistemology

    13. epistemic

      The book below on "epistemic cultures" in science actually led to this reading being considered, so I should mention it here: https://www.amazon.com/Epistemic-Cultures-Sciences-Make-Knowledge/dp/0674258940 It is not a great book, but a great concept. We will discuss it in class in a few weeks.

  9. Nov 2016
    1. Benchmark

      If you are benchmarking others then you are always playing a game of catch-up. Remember, those that you are benchmarking are not sitting still -- they are innovating and move forward. So even when you catch up to where they are today, they are still several steps ahead of you. Not that benchmarking is always bad, you just have to balance it with innovations of your own.

  10. Oct 2016
    1. eorge Panagiotou suggests an i

      It did work. Do you now see this?