403 Matching Annotations
  1. Jan 2019
    1. Banyak kelemahan yang ditemukan Dian ketika menelaah proposal penelitian yang masuk. Ide yang ditawarkan banyak yang kurang kreatif dan aktual. Ada juga yang hanya merupakan duplikasi atau daur ulang dari penelitian sebelumnya.

      Apakah hasil peninjauan ini terbuka untuk umum dan diberikan juga kepada peneliti? Maaf kalau saya keliru, saya setiap tahun mengirimkan proposal ke Kemristekdikti, tapi hasil peninjauan secara lengkap belum pernah saya terima.

    1. The Receptor-like Pseudokinase GHR1 Is Required for Stomatal Closure

      Please find a Peer Review Report here.

      The report shows the major requests for revision and author responses. Minor comments for revision and miscellaneous correspondence are not included. The original format may not be reflected in this compilation, but the reviewer comments and author responses are not edited, except to correct minor typographical or spelling errors that could be a source of ambiguity.

    1. In the context of disaster, social media and other ICTare enablingthe manifestation of a “knowledge commons” [11], a shared information space for victims, onlookers, and the convergent digital volunt

      Cites Elinor Ostrom's work on collective action and commons

      Evokes Benkler et al's work on peer production and commons.

  2. Dec 2018
    1. The norms for using a CSCW system are often actively negotiatedamong users.

      Community norms are well-discussed in the crowdsourcing and peer production literature.

      See: Benkler, Mako and Kittur, Kraut, et al

  3. Oct 2018
  4. Aug 2018
    1. A related series of studies have sought to unpack the dynamics of collabo-ration and to understand which features of peer productions support the cre-ation of higher quality content. This topic has been studied especially closelyin the case of Wikipedia, where particular organizational attributes, routines,norms, and technical features impact the quality of individual contributionsas well as the final, collaborative product.

      Benkler provides examples of studies that examined quality of content as a function of community norms, participant motives, and newbie abuse by experienced editors.

      Has Wikipedia learned anything from these studies? Have they adopted any recommended strategies for improvement? What are the design implications for addressing these issues.

      More here from INFO 5501 reading responses:

      https://drive.google.com/drive/folders/1H0_DTmOspYZ3EwDJGVkBU2RaFeiibr6w?ogsrc=32

    2. A more fruitfulapproach considers variation in peer production success to understand whenand where it works better and worse.

      Benkler's examples of quality studies of large peer groups seems focused on community evaluation of the output rather than what constitutes a high-performing community (process) or a quality values (norms).

      Note: He cover process and norms several paragraphs later.

    3. Just as peer produced goods vary in their nature and form, there is alsoenormous variation between and within projects in terms of the dimensionsalong which quality might be evaluated. In the case of Wikipedia, scholarshave assessed the encyclopedia in terms of factual accuracy, scope of coverage,political bias, expert evaluation, and peer evaluation – often drawing differentconclusions about the quality of Wikipedia or particular articles.

      The quality of Wikipedia articles can vary considerably. Studies point to uneven socioeconomic/cultural/gender/language representation with the ranks of editors.

      The consensus view is that Wikipedia topics are driven by editor interests which results in variations in coverage.

    4. Although this hasconstituted an inconvenient fact in peer production practice, it also reflects animportant opportunity for future research. By focusing only on the projectsthat successfully mobilize contributors, researchers interested inwhenpeerproduction occurs or the reasonswhyit succeeds at producing high qualityoutputs have systematically selected on their dependent variables. An impor-tant direction for peer production research will be to study these failures.

      Failed peer production projects offer potentially interesting insights and should be studied.

    5. Infollowing these three paths, scholars have begun to consider variation withinpeer production projects to understand when and why peer production leadsto different kinds of high quality outputs

      Recent quality studies have explored projects that: • have not attracted sufficiently large communities to wash out bias/inaccuracies, • large communities that have not functioned to create quality information, • different measures/definitions of quality

    6. Both for-profit andnon-profit organizations that have incorporated peer production models havethrived in the networked environment, often overcoming competition frommore traditional, market- and firm-based models.

      But is this a matter of quality or satisficing a need with a free, easily accessible public platform?

    7. Recent work hasbegun to probe more deeply into different dimensions along which qualitycan be conceptualized and measured. This new scholarship has given rise toa more nuanced understanding of the different mechanisms through whichhigh quality resources arise, and founder, in peer production.

      Benkler notes that output "quality" was the focus of early research and has evolved to exploring how it is "conceptualized and measured." Defining and understanding information quality is also connected to my crowdsourcing work.

      However, I'm curious if quality studies also look at the process, in addition to the output.

    8. Peer production successfully elicits contributions from diverse individu-als with diverse motivations – a quality that continues to distinguish it fromsimilar forms of collective intelligence

      Benkler makes a really bold statement here about how peer production differs from collective intelligence. Not sure I buy this argument.

      Brabner on crowdsourcing:

    9. Resolving the tensions between different motivations and incentives presentsa design challenge for peer production systems and other collective intelli-gence platforms. The complex interdependence of motivations, incentive sys-tems, and the social behaviors that distinct system designs elicit has led Krautand Resnick (2012) to call for evidence-based social design and Benkler (2009,2011) for cooperative human system design.

      Benkler cites research where incentives clash re: "material and prosocial rewards". Also, motivations can be temporally-based which demands flexibility in the incentive system as participants' reasons to contribute change and habits/practices/norms become entrenched.

    10. Evidence from this newer body of research shows that motivations are di-versewithincontributors and that different contributors have different mixesof motivations.

      Because motives are diverse and often entangled between intrinsic and extrinsic motives, as well as within/between different groups of participants, designing incentive systems is tricky. Recent research has found that impacts/effects of one type of incentive can't be separated from impacts/effects on other motivational drivers.

    11. The most important insight provided by some of this newer workis that contributors act for different reasons, and that theories based on a sin-gle uniform motivational model are likely to mischaracterize the motivationaldynamics.

      Field and lab experiments have found that motives are not uniform, are complex, vary due to contextual factors, involve social signaling, and have some temporal qualities.

    12. In particular, these neweraccounts have focused on social status, peer effects, prosocial altruism, groupidentification, and related social psychological dimensions of group behavior

      Again, tracking with organizational studies, intrinsic and extrinsic social psychological characteristics have been the focus of more recent work exploring motivations.

      See: https://drive.google.com/drive/folders/1H0_DTmOspYZ3EwDJGVkBU2RaFeiibr6w?ogsrc=32

    13. hat said, a growing number of stud-ies also suggest that these motives interact with each other in unpredictableways and, as a result, are vulnerable to “crowding out” when the introduc-tion of extrinsic incentives undermines intrinsic motivation

      As in the organizational studies of peer production, motivation studies have been conducted increasingly through ethnographic observational and field studies.

      Benkler notes that the varied rationales and patterns for participating in peer production are not singular, and "interact with each other in unpredictable ways."

      Intrinsic motivations (internal rewards) tend to give way to extrinsic motivations (external rewards or consequence avoidance)

    14. Other foundational research on motivation in peer production by Lernerand Schankerman (2010) and others has explored why organizations, firmsand governments, rather than individual users, choose to participate in opensource software.

      More recent motivational studies have focused on organizations' motives for engaging in FLOSS projects as a means to innovate, build knowledge/learning capacity, diversify sources and collaborate.

    15. Despite their differences in emphasis,scope, and genre, all of these surveys support the claim that motivations inpeer production are diverse and heterogeneous.

      Survey studies are widely used in peer production research.

      Observational/ethnographic have also been used to study participants. Results also reflected that motives were varied but also seem to indicate that participants self-selected their projects, collaborators, and specific production roles.

    16. Frequently cited motivations in foundational work by theseauthors, von Hippel and von Krogh (2003), von Krogh (2003), and others in-cluded: the use value of the software to the contributing developer; the hedo-nic pleasure of building software; the increased human capital, reputation,or employment prospects; and social status within a community of peers.Other early accounts analyzing examples of peer production beyond FLOSSsuggested additional motivations. For example, Kollock (1999) emphasizedreciprocity, reputation, a sense of efficacy, and collective identity as salientsocial psychological drivers of contribution to online communities and fo-rums.

      Per Benkler, social psychology constructs (individual behavior, feelings, and thoughts within a social context) offer better descriptions for understanding peer production motivations than economic theory.

      Cited studies in this passage are from:

      CSCW (Beenen, HICSS (Forte and Bruckman), ACM (Nov) GROUP (Panciera) Psychology (Rafarli and Ariel) Social Psychology (Cheshire) CMC (Cheshire and Antin) Law (Benkler) Open Source (Coleman and Hill) MIS (von Krogh et al)

    17. A second quality of peer production that challenged conventional economictheories of motivation and cooperation was the absence of clear extrinsic in-centives like monetary rewards. Traditional economic explanations of behav-ior rely on the assumption of a fundamentally self-interested actor mobilizedthrough financial or other incentives. In seeking to explain how peer produc-tion projects attract highly skilled contributors without money, much of theliterature on peer production has focused on questions of participant moti-vation.

      Peer production contrasts with other forms of labor in its varied non-monetary/economic incentives. Early research on participant motives was grounded in longstanding economic theory/frameworks about self-interested actors.

      The economic approach makes sense, however. Without prior work in peer production, attempting to apply/extend other labor frameworks would be an appropriate evaluation technique.

    18. A newer wave of work has stepped back from this approachand sought to explain how multiple motivational “vectors” figure in the cre-ation of common pool resources online – an approach that underscores a coreadvantage of peer production in its capacity to enable action without requir-ing translation into a system of formalized, extrinsic, carrots and sticks.

      Recent studies of motivation to participate in peer production consider a broader range of decentralized incentives.

      See Kraut et al (2012): http://wendynorris.com/kraut-et-al-2012-building-successful-online-communities-evidence-based-social-design/

    19. Future workcan also begin to address questions of whether, how, and why peer produc-tion systems have transformed some existing organizational fields more pro-foundly than others. An empirically-informed understanding of when andwhere organizational practices drawn from peer production provide efficientand equitable means to produce, disseminate, and access information can pro-vide social impact beyond the insights available through the study of any in-dividual community

      Future directions, suggested by Benkler.

    20. By structuring design changes as exper-iments, these studies make credible causal claims about the relationship oforganizational structure and project outcomes that previous work struggledto establish. By intervening in real communities, these efforts achieve a levelof external validity that lab-based experiments cannot

      The paper suggests that field experiments and intervention studies could offer new insights.

    21. Similar cross-organization studies in other areas of peer production, orstudies comparing across differenttypesof peer production, have remainedchallenging and rare. One difficulty with comparative work across organiza-tions, in general, is designing research capable of supporting inference intothe causes of organizational success and failure

      Benkler also points to a lack of "publicly-available large-scale comparative datasets for types of peer production projects outside of FLOSS" for a reason few comparative studies have been attempted.

    22. Although some of the earliest theories of the organization of peer pro-duction celebrated the phenomena as non-hierarchical, more recent work hasquestioned both the putative lack of hierarchy and its purported benefits (e.g.,Kreiss et al., 2011).

      Later foundational work focused on hierarchies within the various community structures — in contrast to the early perception that peer production was non-hierarchical/anarchistic.

      Benkler suggests that peer production uses a different form of governance and a lighter-weight hierarchical structure than other types of organizations -- not that these groups are anti-hierarchical.

      Cites Keegan's work on gate-keeping in peer production.

    23. More recent work on organizational aspects of peer production has begunto question the “stylized facts” that prevailed in earlier research.

      The second wave of peer production research includes exploration of community attributes and the work they produce, comparative analysis, and theoretical articulations.

    24. This research built on earlier work showing that contrib-utors to bulletin boards, newsgroups, forums, and related systems adopteddurable “social roles” through their patterns of contribution (e.g., Fisher et al.,2006) – an approach that was subsequently applied to Wikipedia by Welseret al. (2011) and McDonald et al. (2011)

      Curious why Benkler didn't cite Q&As and group blogs as peer production cases, e.g., Metafilter, DailyKos, etc., which operate quite differently than the examples listed here.

    25. A large body of descriptive work has also sought to characterize organi-zational dimensions of Wikipedia and the practices of its contributors.

      The cited Wikipedia literature intersect a number of focus areas including qualitative studies od participation, production process, governance, onboarding and socializing activities, leadership and quantitative/inductive studies of the organization.

      Again, is the number and depth of citations here simply a result of Benkler's interest/knowledge area? Or is there something fundamentally different between FLOSS and Wikipedia?

    26. Much of the earliest empirical research inductively sought to describe thestructure and organization of FLOSS communities

      The FLOSS literature that Benkler cites seems to be much more focused on organizational structures than the Wikipedia study examples in the next paragraph.

      I'm curious why that is the case? Is there a wider body of related work than Benkler is citing? Maybe the FLOSS production process is less accessible than Wikipedia which is entirely online and embedded in its site. Is this a convenience issue more so than a diverse interest issue?

    27. Other foundational accounts, like Moglen (1999) and Weber (2004),attended to the emergence of informal hierarchies and governance arrange-ments within communities.

      Other early work focused on organizational attributes, like how information goods were produced with flat/informal hierarchies, community values/norms, and governance structures.

    28. Peer productioncould, Benkler argued, outperform traditional organizational forms underconditions of widespread access to networked communications technologies,a multitude of motivations driving contributions, and non-rival informationcapable of being broken down into granular, modular, and easy-to-integrate

      Benkler's early work studied "the role of non-exclusive property regimes and more permeable organizational boundaries" for knowledge products.

    29. Initial scholarship on the organization of peer production emphasized the-oretical distinctions between peer production, bureaucracy, and transactioncost explanations of firms and markets.

      Description of early research

    30. Indeed, peer production com-munities perform all of the “classical” organizational functions like coordina-tion, division of labor, recruitment, training, norm creation and enforcement,conflict resolution, and boundary maintenance – but do so in the absence ofmany of the institutions associated with more traditional organizations.

      List of typical activities enacted in peer production communities. Early research sought to identify these activities and the unique nature of peer production work. Later studies examined the effectiveness of peer production and how the activities as communities mature.

      Benkler notes later in the same passage:

      "However, as peer production communities have aged, some have acquired increasingly formal organizational attributes, including bureaucratic rules and routines for interaction and control."

    31. Although peer production is central to social scientific and legal researchon collective intelligence, not all examples of collective intelligence created inonline systems are peer production. First, (1) collective intelligence can in-volve centralized control over goal-setting and execution of tasks.

      Not all collective intelligence is peer production.

      Peer production must adhere to values: de-centralized control, broad range of motives/incentives and FLOSS/creative commons rights.

    32. Peer production is the most significant organizational in-novation that has emerged from Internet-mediated social practice, among themost visible and important examples of collective intelligence, and a centraltheoretical frame used by social scientists and legal scholars of collective intel-ligence.

      Benkler ranks peer production's place in the social science literature on collective intelligence.

    33. Following Benkler (2013), we define peer produc-tion as a form of open creation and sharing performed by groups online that:set and execute goals in a decentralized manner; harness a diverse range ofparticipant motivations, particularly non-monetary motivations; and sepa-rate governance and management relations from exclusive forms of propertyand relational contracts (i.e., projects are governed as open commons or com-mon property regimes and organizational governance utilizes combinationsof participatory, meritocratic and charismatic, rather than proprietary or con-

      Peer production definition per Benkler.

    1. se durability of data.

      Would be good to give a basic list either here or in the above where you introduce metadata, or even a filled-out dummy form that shows some basic metadata. Would help folks who still have trouble visualizing the boundary between metadata and paradata.

    2. a Plan

      Storage and Preservation

    3. Deprecated formats

      Is this what header 2 looks like? Would be good to format these items so they stand out. The text is really understandable and helpful though.

    4. ld.

      Here introduce the key concepts you define below by saying something like: in order to get as close to a perfectly usable and preservable dataset, you should collect both metadata and paradata.

    5. The

      Before this sentence, introduce paradata somehow, like: "What you can do is record paradata, or information about how and why the data was created, like a data diary."

    6. ad infinitum

      This is a bit overwhelming - are you saying you should not aim to collect "all the data there are" and set appropriate boundaries, or that you should not expect to avoid all possible hiccups (a "perfect" dataset)? Or both? I think since you used the previous paragraph to point out some specific concerns to address, you should explicitly reference them here. I think you're saying that the key to good datasets is having reasonable and attainable goals, rather than striving to get "all the data and complete in every way possible".

    7. What do I want my data to do?

      This section is starting to get a bit hard to understand because the data are taking on person qualities. Could you introduce an example of an amusing fake-name project so that folks who do not discuss data in the abstract could more easily follow along?

    8. .

      :

    9. The

      "It is important to plan the structure of the data, expectations for its use by researchers, and arrange for its storage from the beginning of your research."

    10. You’ll notice that quality data (however you define the term) take planning.

      "Creating good data requires planning"

    11. As an added benefit, those same qualities that make data useful and conclusions reproducible can also help to protect against the ravages of time

      "And maintaining data in this condition helps protect them from the ravages of time" maybe?

    12. The data might be saved as an image, when in fact it's got a structure.

      Reword this so it is clear that it is different from the above: Access part = text and numbers and database saved in a bad format but still usable // Reuse part = a map or output saved as an image but the underlying data used to create it is not present (did I understand correctly?)

    13. proprietary

      Make sure somewhere earlier in the text someone has defined "proprietary" in plain language

    14. do

      ?

    15. What would make it better?

      Remove this question - the following section needs to be read first otherwise it is overwhelming

    16. in the past when trying to understand

      remove "in the past" and change following to "when trying to use someone else's data or reconstruct their conclusions"

    17. ef back to “The first steps in going digital are quite easy. They are fundamentally a question of maintaining some basic good habits. Everything else flows from these three habits:” section 1.1.6. These principles will help you and future researchers use your data, reproduce your conclusions and “future-proof” your digital work.

      I assume this is going to be turned into a link to a previous chapter?

    18. great

      Remove

  5. Jul 2018
    1. We’ve run into roadblocks, and people really appreciate hearing about them because for the most part they’re running into the same issues
    1. My confidence also has come from the people I’ve surrounded myself with. I’ve been lucky enough to be part of honors programs in schools and worked in engineering and coding programs. I’ve also been a part of so many communities that are all full of determined, articulate, well-educated people who want to support each other and learn from each other. Because of that, I have grown to absorb some of those qualities myself.
    2. It creates more of a community because you get closer to someone once you’ve spent a solid 24 hours working with them. It’s definitely experience to be had.
  6. May 2018
    1. “OER are not typically counted toward research requirements, because they are seen as lacking the vetting process that comes with, for example, peer-reviewed articles.”
  7. Mar 2018
    1. In what appears to be a first, a U.S. court is forcing a journal publisher to breach its confidentiality policy and identify an article's anonymous peer reviewers.

      Wow. This could have a chilling effect on reviews for certain subjects.

    1.  By asking my students to craft and peer-review multiple-choice questions based on the concepts covered that week (and scaffolding this process over the semester)

      Este párrafo muestra el "ingrediente" de los 8 ingredientes de la pedagogia abierta de Peer review al hacer que los alumnos colaboren en el proceso de evaluación.

  8. Feb 2018
    1. Behind all the things on the panel is a pinkish/peach  layer.

      I would avoid using the words "things."

    2. Lettering It spells out the word Mitchell David Mucha M.D., in white with a pinkish/peach boarder. The stitching on the words is very rough. The letters are very huge and take up a majority of the space on the panel.

      I would maybe try to add transition sentences to more smoothly transition between ideas.

    3. idea of the Stethoscope was from René Théophile Hyacinthe Laënnec

      Good job including links and images on the page to give the reader a fuller experience.

    4. This long part of it is the same color as the bag except it has more of  rough touch to it.

      There is a couple grammar errors. Here there should be an "a" before "rough." "...more of a rough touch to it."

    5. About The Panel

      Before describing the panel, I would give the audience some background information on The Quilt, such as the founding or what each panel means.

    6. The orginal Doctor’s Bag was the Gladstone which was made in the mid nineteen century by J.G. Beard. It was used for house visits to patients house. The contents inside of the bag is medical tools like stethoscope, clinical thermometer and tongue depressor some form of illumination, such as a torch, plessor, ophthalmoscope and auriscope; a test tube or two; and bottles of Benedict’s reagent and acetic acid to complete the kit (RACGP).

      Good job giving some background information the Doctor's Bag.

    7. The items that I will be describing from the panel is a doctor bag, stethoscope, the colors, and finally the lettering. I will be describing them in the same order I have mentioned them.

      I would avoid writing in the third person. (Not using "I")

    1. The top center panel, belonging to Eddie (no last name reported), has a mosaic background of 6″x 6″ burgundy, soft dusty rose, light bubblegum pink, and sapphire blue squares. His name is then sewn in large, cursive lettering across the top left half of the panel.

      This description makes me feel as if I'm looking at the panel. Love it

    2. Since the panels would be featured in the Quilt as a visual memorial and not as a blanket, I wondered why the panels that were predominantly paintings were not made of canvas fabric instead of fabrics associated with apparel, or at least primed with some kind of Gesso to preserve the piece. I am by no means an expert, but as an artist who has experimented with different mediums on both primed and un-primed fabrics, I can attest for the value of using the right mediums on their respective materials. Though I am sure acceptable fabric paints were mostly used, I could tell where they were not.

      Great connections between your experience as an artist and what you've observed from the quilt. I enjoy the objectivity rather than simply taking the panel for what it is.

    3. Although tied by a similar tragedy, each panel exhumes individuality through applying different artistic methods.

      The individuality of the panels is nicely described and understood. From the previous description, there is distinct differences yet similarity in the pieces.

    4. Each panel is made of a soft fabric and sewn onto a large, 12’x 12′ piece of ivory linen fabric.

      This first description of the panel gives a brief but detailed imagery. Very good

    5. Block #621

      Great pictures - but could be more multi modal as a whole

    1. the rainbow in the body of the image is shown vibrantly in the majority of picture which by its colorful characteristics

      Was there any writing? Any markings? Differences in stitches? Add more details, even mentioning the lack of detail ( Ex: stating the panel has no visible markings...) to give the reader a fuller image.

    2. Another important visual element that I detected within this image was the blue bracelet circling the shiny arm holding the dog.

      Were there any additional objects (letters, pictures, notes,ETC) that came with the panel? Maybe there will be some clues as to what the dog or bracelet means.

    3. This gives the rainbow an ever greater meaning, maybe Jimmy had an aspiration for music and pursued a career in one, or possibly just has a respect for the fine musical arts.

      From reading this I can tell you are really starting to question and investigate this panel as well as, Jimmy Popejoy.

    4. hings within the panel that I took notice of immediately was the dog that was held by the shiny arm.

      I like that you started with the most attention grabbing thing to you. However, I think you should first introduce The AIDS Quilt. That way the reader will understand what it is.

    5. The panel base is mostly a pure, bloody, vibrant red color. This is also in the material of somewhat a shiny, soft, velvet material that vividly gives the panel some extravagant flare.

      Love the word choice, the description brings a vivid image to mind and flows smoothly.

  9. Nov 2017
    1. Media Maker Spaces is an exploration for them experience a creating, editing, storing. publishing and streaming media to their peers and to their immediate context.

      Seems quite related to #CollecionAndIdentity

    1. Festivals and rituals - Living archives of the memories

      Seems quite related to #CollectionAndIdentity

    1. The aim is to demonstrate the distance travelled on their journey in the form of tangible, trackable learning outcomes and applications.
    1. An excellent commentary on what ails our current peer review system and how alternative quality assurance system might work in academics.

  10. Oct 2017
    1. review and critique each other’s work.

      This is the process of replying to annotations. But annotation can also be leveraged for peer review of student writing.

    2. listen well—to be a good “critical friend.”

      Read classmate's annotations, respond appropriately: respectful, challenging...

    1. Brianna:I had a negative experience where, in my master’s, my supervisor encouraged me to submit one of my papers to a journal for publication. I just submitted the paper to a journal as a course paper without making any changes, not even changing the title page. The journal told me to re-submit with revisions, but I thought thatit was a rejection, and I stopped the process—it was intimidating. I thought being involved in a journal where I know some of the people and they won’t just get an online e-mail response from editors would be helpful

      Misunderstanding revise and resubmit; misunderstanding the difference between a student paper and an article.

  11. Sep 2017
    1. The problems here stem from a lack of comprehensiveness, interoperability, and critical mass uptake as the de facto platform for PPPR. The result of this is a mess of different platforms having different types of commentary on different articles, or sometimes the same ones, none of which can be viewed easily in a single, standardised way. That doesn’t seem very efficient.

      This is really key.

  12. Jul 2017
    1. "What our collaborative learning style empowers and enables is a student's resilience -- how do you look to your neighbor as a resource, how do you test your own theories, how do you understand if you're on the right track or the wrong track?" says Monique DeVane, College Prep's head of school. "It teaches them that it's not just about content; it's about cultivating habits of mind that are the underpinnings of deeper scholarship."
  13. Jun 2017
    1. protected platform whereby many expert reviewers could read and comment on submissions, as well as on fellow reviewers’ comments

      Conduct prepeer review during the manuscript development on a web platform. That is what is happening in Therapoid.net.

    2. intelligent crowd reviewing

      Crowdsourcing review? Prepeer review as precursor to preprint server.

  14. Apr 2017
    1. p. 1

      Peer review is a mechanism, then, for quality control; It protects us from contamination by error and poor argument, and affords us truth or contributions to attaining truth.

    2. Shatz, David. 2004. Peer Review: A Critical Inquiry. Issues in Academic Ethics. Lanham, Md: Rowman & Littlefield.

    1. The Effects of the Built Environment on Child Friendliness and Obesity: Analysis on Auburn Ave

      Charmaine's claim isnt clear at this very moment but she provides alot of background information on her location that I didnt knw about. If she specifically points out her claim them this analysis would be good.

    1. Conclusion

      Claim: By investing in communities through new businesses and innovations, the population demographic may experience a drastic change from low income residents to high income residents. This then leads to other benefits such as a lower crime rate.

      Evidence: The author uses a chart to display the distribution of income when looking at ages in the Ponce de Leon area (secondary source).

      Overall, I really enjoyed reading your BEA. I felt that it was professionally crafted with the rhetorical situation kept in mind. One addition I would suggest would be looking also at the negative aspects of of gentrification, such as the displacement of residents that resided in the area before the process of gentrification began. By doing this, it would add to your credibility by recognizing and arguing the opposing position.

      GREAT JOB!!!!!

    1. Description of Ponce City Market

      Like the last page, the current description of Ponce City Market provides strong evidence for the claim of how gentrification can improve an area in various ways. Be sure to actually state your ideas to reinforce your ideas. Think about once again addressing the questions as well.

      The pictures may also need to be cited if not personally captured.

    1. Description of Krog Street Market

      Claim: The introduction claim needs to be incorporated into this description, because it directly provides evidence for such ideas.

      Evidence: By using a highly detailed description as well as pictures, it's clear that the Krog Street Market has experienced obvious improvements. Use the questions suggested at the beginning to further emphasis how much the area has improved for the better!

    1. Demographics of Ponce de Leon Area

      Claim: Like previous pages, the claim was not explicitly stated, but the information implied a change in population over the years.

      Evidence: The chart was utilized in this case form outside sources cited both at the bottom of the page as well as within the conclusion. As I previously mentioned, I would include more about income, education, and business statistics, because I feel that this would relate better to your claim in the introduction.

    1. Demographics of Krog Street Area

      Once again, this is great evidence for your claim in the introduction.

      Claim: Your introduction claim of gentrification could really be used here to drive the point home. (The claim was implied but not stated.)

      Evidence: For evidence, secondary sources were utilized to describe the changing demographic. I would suggest to discuss in further detail the categories of income and business. These categories would provide a great deal of validity to your claim, should you include it.

      BE SURE TO CITE YOUR CHART

    1.  (“725 Ponce” 2015)

      I would personally consider putting the full citation at the bottom of each page. I feel that this would be more helpful and easier to access for your audience.

    2. History of Ponce City Market

      Wow! This is great information. You did a great job constructing the history of Ponce in a way that is easy for the reader to understand and enjoy. I also really like the vintage pictures!

      Claim: Like the previous section on the history of Krog Street Market, there is no claim physically present in this paragraph. To me, it is easily understood to be in support of the introduction claim, but you may want to explicitly state how Ponce City Market relates to the claim for the purposes of the project.

      Evidence: The evidence is taken from secondary source cited both at the bottom of the page and the conclusion. Once again, I would consider addressing the questions I suggested in the previous annotation as a way to address the claim implied.

    1. History of Krog Street Market

      I really enjoyed reading the history of this particular location. Surprisingly, I actually remember when this property was purchased and the media attention that followed on the local news channels.

      Claim: There wasn't a claim explicitly stated in this paragraph but the claim in the introduction was obviously implied.

      Evidence: This paragraph mainly consisted of second hand information sited in the conclusion. I personally believe that learning about the history of the market and the surrounding area adds to your authenticity and credibility.

      It maybe helpful to restate the claim and provide further evidence such as what the area looked like before versus years after the purchase. Was it run down? Also relating back to the introduction what were the crime rates before and after at this location? What was the local economy like before the recent innovations? Overall, I feel that this is a strong paragraph that could be improved through relating more to the claims in the introduction.

    1. Introduction

      I really enjoyed your introduction. I feel that discussing what a built environment is and how it came about improves the readability for your audience. Further, your introduction highlights your claims about gentrification in Atlanta! I'm excited to continue reading!

      Claim: When lower income neighborhoods experience gentrification, the population demographic undergoes drastic change (i.e. higher incomes). In turn, this may lead to positive benefits for the economy as well as the safety and happiness of residents.

      Evidence: There is no evidence in this specific paragraph, but I assume that the other pages on Ponce City Market and Krog Street Market will act as examples. Maybe give a brief introduction of both in order to provide evidence in this paragraph.

    1. Thesis: The rhetoric of the built environment of Atlanta shows that racial discrimination, white flight, car dominated transportation network, and segregation by race and class have caused Atlanta to have the highest income inequality ratio in the country, and the same factors that led to severe income inequality in Atlanta are perpetuating the problem today.

      The author makes the claim that Atlanta has a preeminent car based transporation system and that race, class, and racial discrimination has been a determinant in income inequality in Atlanta, stating it to be the highest in the country. Providing data or a referenced source would help to give clarity to Atlanta's ratio compared to all other countries in the U.S.

      The authors' thesis is very thought-provoking and descriptive, but can be broken up into two-three sentences.

    2. Built Environment Analysis (DRAFT)

      Overall, the author does a great job in providing claims and arguments, but the draft in totality is not complete and needs the inclusion of citations from sources and data, graphs, etc. to provide evidence for the claims that are being made and also for general reference when mentioning ratios and numbers. Also, the author did not incorporate multiple modes of presentation. To meet the requirement, the authour should provide photographs, videos, charts, etc. which will also help to give stated claims and arguments a varied perspective.

    3. As a result of a long history of white flight and racial discrimination, Atlanta’s transportation network is predominately designed for travel by car. Consistent public transportation is present downtown and in the immediately surrounding areas. Evidence:

      This section makes the claim that Atlanta has an intentional, leading car based transportation system and that it has a connection to past racial segregation.

    4. Cost of living map and MARTA map side by side

      Listed here seems to be a description of two photographs that are to be compared, however there are no photograpghs posted to compare anything. The author should provide those images for reference and comparison as well as a description on what is being compared and why.

    5. The neighborhood one grows up in has been shown to impact their chances for upward economic mobility, therefore gentrification and neighborhoods segregated by class perpetuate income inequality.

      The author makes the claim that the neighborhood of ones upbringing has a direct correlation to their future rank in the economic sphere of the world. Examples on how this statement may be accurate should be provided in this area.

    6. The Fading American Dream: Trends in Absolute Income Mobility Since 1940” by Raj Chetty and Nathaniel Hendren “All Cities Are Not Created Unequal” by Alan Berube

      The author provides a list of evidence based sources, but does not provide key details in how the sources benefit the claim. Author needs to provide citations and references from sources to support the claim that is being made

    7. history of white flight

      Listed here is a term that was unfamilar and provided research shows that there is a book entitled by this name White Flight by Kevin M. Kruse: http://press.princeton.edu/titles/8043.html

      Author could provide a brief description of the term because it will enhance this specific claim amd also include the link to the book's overview and how it possibly relates to the claim.

    8. While there is some public transportation for people living further away from the center of the city, the current accommodations are insufficient for people without cars.

      The author makes the claim that exisitng public transportation does not benefit persons without a vehicle, but acknowledges its benefits to persons that are located in somewhat distant locations from Atlanta.

    9. The trend of Atlanta’s middle and upper classes moving out to the suburbs is shifting, and these groups are beginning to move back into the city. Therefore, neighborhoods are being gentrified to meet the growing demand.

      The author makes the claim that Atlanta neighborhoods are becoming more gentrified due to the incoming masses of middle class persons/families.

    10. Low-income residents that have settled close to the city, along public transportation routes, are having to move further out because the gentrification of neighborhoods raises the cost of housing.

      Author's claim is gentrification and higher property taxes in Atlanta neighborhoods have caused perons/families of lower incomes to leave cities allowing middle-class persons/families to move in. Specific examples of this scenario in scecific Atlanta areas should be incorparted into the author's claim and provided for reference.

    11. The quality and quantity of public transportation decreases as you move further away from the center of the city. Consequently, those living in poverty who have relocated further away from the city are in a worse situation because they do not have the same amenities available to them.

      This claim explains the disadvantages that lower-income person families face when leaving areas that provide abundant access to public transportation and move to areas with little to less public transportation prone to be of lower quality. An example of this case should be provided in this area to compare public tranportaion between a city area and an area in which it lacks.

    12. “Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “Health Impact Assessment of the Atlanta Beltline” by Catherine Ross

      This is a list of the sources that the author included to support the claim, but there are no specific details provided to explain how or why it benefits the argument being made.

      The author should provide direct, specific evidence from the listed sources to support the claim.

    13. “Using Vehicle Value as a Proxy for Income: A Case Study on Atlanta’s I-85 HOT Lane” by Sara Khoeini and Randall Guensler “Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “The Human Scale” by Andreas Mol Dalsgaard

      This is a list of the evidence based sources for the authors claim. The author should add links to the sources and provide specific evidence and citations from the sources that pertains to the claim that is being made.

    14. Photo of bench in Little Five Points

      The author makes note of a photograph that is not posted so there needs to be a photograpgh in this area as well as a description on its relation to the claim.

    15. “Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “How Cities Use Design to Drive Homeless People Away” by Robert Rosenberger

      Here are the sources provided to accomodate the author's claim, but the evidence to back up the claim in not provided.

    16. “CHANGING BOHEMIA Little Five Points, a Haven of Counterculture, Faces Gentrification and Dissension” by Melissa Turner “Health Impact Assessment of the Atlanta Beltline” by Catherine Ross

      Sources for this claim focus on the Atlanta Beltline and the Little Five Points Area, however, the author does not include any major points or refernces to enhance the claim on gentrification in Atlanta areas directly. An incorporation of in-text citations is needed.

    17. “Atlanta: Unsafe at any Speed: Transit Fatality Raises Issues of Race, Poverty and Transportation Justice” by Laurel Paget-Seekins “The Human Scale” by Andreas Mol Dalsgaard

      Provided here are a list of evidence based sources to reiterate the authors claim.

      Again, the author should add links to the sources and provide specific evidence and citations from the sources that pertains to the claim that is being made.

  15. Mar 2017
    1. ittle direct indication that the Trump administration or congressional leaders known for attacking scientific research on climate change and human health are looking to exploit reproducibility campaigns as a political opportunity.

      Easy to connect these dots though.

    1. Eve Marder, a neurobiologist at Brandeis University and a deputy editor at eLife, says that around one third of reviewers under her purview sign their reviews.

      Perhaps these could routinely become page notes?

    2. If Kriegeskorte is invited by a journal to write a review, first he decides whether he’s interested enough to review it. If so, he checks whether there’s a preprint available—basically a final draft of the manuscript posted publicly online on one of several preprint servers like arxiv and biorxiv. This is crucial. Writing about a manuscript that he’s received in confidence from a journal editor would break confidentiality—talking about a paper before the authors are ready. If there’s a preprint, great. He reviews the paper, posts to his blog, and also sends the review to the journal editor.

      Interesting workflow and within his rights.

    3. The tweet linked to the blog of a neuroscientist named Niko Kriegeskorte, a cognitive neuroscientist at the Medical Research Council in the UK who, since December 2015, has performed all of his peer review openly.

      Interesting...

  16. Feb 2017
    1. object you are photographing b

      "Maybe give some examples..."

    2. I hope to show with this tutorial, however, that the Dino-Lite Premier AM-311S has the potential to create useful models at an affordable price.

      "Good introduction of scope..."

    3. capture images for processing

      "Maybe be more specific..." (generic commentary)

    1. The struggle between Whewell and Lubbock represented two distinct visions of what a referee might be. Whewell was the authoritative generalist, glancing down on the landscape of knowledge. He was unconcerned with — and probably not in a position to critique — the details. Such referees were, according to the Royal Society's president, “Elevated by their character and reputation above the influence of personal feelings of rivalry or petty jealousy”4. Lubbock was a younger specialist, Airy's equal. This allowed him to take a fine-tooth comb to Airy's arguments; it also put him in the position of reviewing a direct competitor.

      Two versions of what a review is.

    1. Despite many editors being unpaid or poorly remunerated for their work, plant scientist Jaime A. Teixeira da Silva believes they “should be held accountable” if authors are made to wait for an “excessive or unreasonable amount of time” before a decision is made on their research.

      How would this be enforced exactly?

    1. Pivotal roles are played by three enzymes, (phospho-fructokinase (PFK), pyruvate kinase (PK) and phosphofructoki-nase/fructose-2,6-bisphosphatase (PFKFB)) through their inhibi-tion or activation by three reaction intermediates (fructose-1,6-bisphosphate (F16BP), fructose-2,6-bisphosphate (F26BP), andphosphoenolpyruvate (PEP)) in glycolysis. These enzymes havemultiple isoforms (PFKL/M/P, PKM1/M2/L/R and PFKFB1-4)which are subjected to contrasting allosteric regulations [9–11].Each isoform, therefore, affects the glycolytic activity in a distinctmanner.All three isoforms of PFK are activated by F6P and F26BP [12],but only PFKM and PFKL are activated by F16BP [13–15].PFKFB is a bifunctional enzyme whose kinase and bisphosphatasedomains catalyze the formation and hydrolysis reaction of F26BP,respectively [9,16]. Isozymes of PFKFB differ in their kinase andphosphatase activities as well as in their sensitivity to feedbackinhibition by phosphoenolpyruvate (PEP) [17–19]. Thus, eachisozyme of PFKFB has a profoundly distinct capacity inmodulating PFK activity. Pyruvate kinase (PK) in mammaliansystems is encoded by two genes that can produce two isoformseach. Except for the PKM1 isoform, the other three isoformsof PK, PKM2, PKL and PKR, are activated by F16BP to varyingextents [11]. The M2 isoform of PK, in addition to activation byF16BP, is also under the control of a host of allosteric modulatorsincluding serine, succinylaminoimidazolecarboxamide ribose-5-phosphate (SAICAR) and phenylalanine among others [

      Need a figure presenting the regulation network.

  17. Jan 2017
  18. Oct 2016
  19. Aug 2016
    1. Clarity on what qualifies as a respected preprint

      Why should we respect a preprint? I'm not sure that we should respect anonymously peer reviewed journal articles as much as we do. It's important to remain critical, and I worry that trying to put a veneer of 'respectability' over preprints is not as helpful as expecting people to read them to judge content.

  20. Jul 2016
    1. Page 62

      Borgman discussing the purpose of peer review

      Pre-publication mechanisms serve as expert filters on what becomes part of the scholarly record, when doing out there researchers reading list.

    2. Page 60

      The use of a publication form as a proxy measure for the quality of research productivity has distorted the peer-review system so severely that some consider it broken. Peer reviewing is an expensive process, requiring considerable time and attention of editors comma editorial board members, and other reviewers. Top journals in the sciences and medicine they put fewer than half of the submitted papers through a full P review process, rejecting the remainder on an initial editorial review, and ultimately publish 6 to 10% of the total submissions. Particularly in The Sciences, researchers are under so much pressure to place papers and talk to your journals that they submit them to the same journals, whether or not the content is appropriate.

  21. Jun 2016
    1. No Bias, No Merit: The Case against Blind Submission

      Fish, Stanley. 1988. “Guest Column: No Bias, No Merit: The Case against Blind Submission.” PMLA 103 (5): 739–48. http://www.jstor.org/stable/462513.

      An interesting essay in the context I'm reading it (alongside Foucault's What is an author in preparation for a discussion of scientific authorship.

      Among the interesting things about it are the way it encapsulates a distinction between the humanities and sciences in method (though Fish doesn't see it and it comes back to bite him in the Sokol affair). What Frye thinks is important because he is an author-function in Foucault's terms, I.e. a discourse initiator to whom we return for new insight.

      Fish cites Peters and Ceci 1982 on peer review, and sides with those who argue that ethos should count in review of science as well.

      Also interesting for an illustration of how much the field changed, from new criticism in the 1970s (when the first draft was written) until "now" i.e. 1989 when political criticism is the norm.

    2. Nevertheless, there were a few who questioned that definition of fairness and challenged the assumption that it was wrong for reviewers to take institutional affiliation and history into consider- ation. "We consider a result from a scientist who has never before been wrong much more seriously than a similar report from a scientist who has never before been right. . . . It is neither unnatural nor wrong that the work of scientists who have achieved eminence through a long record of important and suc- cessful research is accepted with fewer reservations than the work of less eminent scientists" (196). "A reviewer may be justified in assuming at the outset that [well-known] people know what they are do- ing" (211). "Those of us who publish establish some kind of track record. If our papers stand the test of time . . . it can be expected that we have acquired expertise in scientific methodology" (244). (This last respondent is a woman and a Nobel laureate.)

      Fish reporting on the minority in response to Peters and Ceci who argued that track records should count in peer review of science

    3. A similar point is made by some of the participants in a discussion of peer review published in the Behavioral and Brain Sciences: An International Journal of Current Research and Theory with Open Peer Commentary (5 [1982]: 187-255). The occasion was the report of research conducted by D. P. Peters and S. J. Ceci. Peters and Ceci had taken twelve articles published in twelve different journals, altered the titles, substituted for the names of the authors fictitious names identified as researchers at institu- tions no one had ever heard of (because they were, made up), and resubmitted the articles to the jour- nals that had originally accepted them. Three of the articles were recognized as resubmissions, and of the remaining nine eight were rejected. The response to these results ranged from horror ("It puts at risk the whole conceptual framework within which we are accustomed to make observations and con- struct theories" [245]) to "so what else is new."

      Peters & Ceci 1982 comes up!

    4. . Predict- ably, Schaefer's statement provoked a lively exchange in which the lines of battle were firmly, and, as I will argue, narrowly, drawn. On the one hand those who agreed with Schaefer feared that a policy of anonymous review would involve a surrender "to the spurious notions about objectivity and absolute value that . . . scientists and social scientists banter about"; on the other hand those whose primary concern was with the fairness of the procedure believed that "[jiustice should be blind" ("Correspon- dence" 4). Each side concedes the force of the opposing argument-the proponents of anonymous re- view admit that impersonality brings its dangers, and the defenders of the status quo acknowledge that it is important to prevent "extraneous considerations" from interfering with the identification of true merit (5)

      Discussion of debate at MLA about plan to introduce blind submission to PMLA and comparison with sciences and social sciences.

    1. ouble-blind) peer review became an established component of thepost-war scientific bureaucracy (Chubin & Hackett, 1990,pp. 19 –24)

      history of peer review

    Tags

    Annotators

  22. May 2016
    1. When students see adults actually listening to them with respect, that is when they begin to realize they have a voice and can make a difference in their world.

      I hope this is true. And I love the idea that adults are that important to students. Still I wonder how this fits with the connected-learning notion that youth want to be heard and recognized by their peers. I suppose it isn't an either/or: some youth seek peer approval, others want to be heard by adults. When you post on an open social network, you never know who will respond.

  23. Apr 2016
    1. Does peer review work? Is peer review broken? The vast majority of authors believe it improves their final work, and since it’s evolving from this solid base, it’s clearly not broken. But before we can have a useful discussion about its purpose and effectiveness, we need to agree on which approach to peer review we’re talking about, then whether our expectations of it are reasonable and accurate.
    2. Here are some variables around peer-review we have to understand before we know what kind of peer review we’re actually talking about: Is it blinded? If it is blinded, is it single-blinded or double-blinded? Is there statistical or methodological review in addition to external peer-review? Are the peer reviewers truly experts in the field or a more general assemblage of individuals? What are the promises and goals of the peer review process? What type of disclosure of financial or other potential competing interests is made? Are reviewers aware of these? Is there a senior editor of some sort involved along with outside peer reviewers? Is the peer-review “inherited” from another body, such as a committee or a preceding journal process (e.g., in “cascading” title situations or when expert panels have been involved)? Are there two tiers of peer review within the same journal’s practices? Is the peer-review done at the article level or at the corpus level (as happens with some supplements)? Is plagiarism-detection software used as part of the process? Are figures checked for manipulation? Is the peer reviewer graded by a senior editor as part of an internal evaluation and improvement process?
    1. White (1984, cited by Vaughan, 1991) reported on a study conducted at California State University in which two essays were tucked into a huge sample of essays and read a year apart by the same readers using a 6-point scale. The reading a year later produced scores that were identical to the first in only 20 per cent of the cases. The scores differed by one point or less in 58 per cent of cases and 2 points or less in 83 per cent of the cases. As White points out, a 1-point difference is generally considered unproblematic, but on a 6-point scale the difference between a 3 and a 4 is the difference between a pass and a fail. Obviously, then, changes in examiner severity/leniency over-time have implications for maintaining standards, and must be monitored. Research has been conducted into variations in examiner severity/leniency during the marking of a particular allocation of scripts, a marking period, and over more extended periods of time.

      intrarater reliability is only 20%

    2. According to Stemler, consistency estimates of interrater reliability assume that it is not necessary for judges to share a common meaning of the rating scale, so long as each judge is consistent in their classifications.

      Wittgenstein's beetle in a box

    3. (2004) notes that most research papers describe interrater reliability as though it is a single, universal concept. He argues this practice is imprecise and potentially misleading. The specific type of interrater reliability being discussed should be indicated. He categorises the most common statistical methods for reporting interrater reliability into one of three classes: consensus estimates; consistency estimates; and measurement estimates.

      Stemler 2004

    1. in the latter both the wide differential in manuscript rejection rates and the high correlation between refereerecommendations and editorial decisions suggests that reviewers and editors agree more on acceptance than on rejection.

      In "specific and focussed" fields, the agreement tends to be more on acceptance than rejection.

    2. In the former there is also much more agreement on rejectionthan acceptance

      In "general and diffuse" fields, there is more agreement on paper rejection than in "specific and focussed."

    3. . Referees ofgrant proposals agree much more about what is unworthy of support than about what does have scientific value. In

      Grant referees are better at agreeing on inadequate work than adequate

  24. Mar 2016
    1. Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H. D., & O’Mara, A. (2009). Gender effects in the peerreviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevelapproaches.Review of Educational Research, 79, 1290–1326
    1. One mid-career scientist told a story of how he and others in his lab counteractedan abuse of power by his mentor, a senior scientist, while he was in training. Hismentor received a manuscript to review that was authored by a ‘‘quasi-competitor.’’It presented results of experiments similar to those that were going on in thementor’s lab. The scientist continued, ‘‘That paper ... basically would have beat usto the punch. They would have published these results before us, and they wouldhave gotten credit, and not us. And my mentor, God bless him, sat on the paper.’’The mentor not only delayed writing the review but asked someone working in thelab to write it (a move of questionable ethicality in itself). That lab person and ourrespondent decided, in response, to stall their own work, so that their lab would nothave an unfair advantage over the group who submitted the paper for review. In theend, the original group got credit for the findings, while the respondent’s lab wasalso able to publish their slightly different findings. He ended his story with,‘‘Sometimes you’re in an awkward position, and you try to do the best thing you canunder the circumstances, within your own internal ethical clock or whatever. Andsometimes it’s ugly and it’s imperfect, but it’s the only thing you can do. If we hadgone to the mentor and voiced this objection, our careers would have been over. Ifwe had approached the journal—God forbid, forget it.’’ The speaker qualified thisstory by saying that it made him sound much more ethical than he actually is.

      peer review deliberately delayed in order to slow competitor

    2. The focus-group discussions showed, however, that scientists see peer review asaffording a unique, even protected opportunity for competitors to take advantage ofthem. In this sense, competition infects the peer review process, not only throughscientists’ competition with other applicants, but also through scientists’ distrust ofthe reviewers themselves, as competitors. The following exchange among mid-career discussants shows their sense of vulnerability

      Evidence that peer-reviewers are competitors.

    3. I really hate to admit this, but you do the same thing with your competitors asyou do with grant agencies. You sucker-punch them. You might have—when Isubmit a paper, I already have the next two or three papers’ worth of data. Imean, I know what I’m going to be publishing a year from now, mostly. But thepaper that comes out of my lab is Part A. Parts B and C are mostly on my desk.And I’ve put things in part A to basically entice my competitors into making anass out of themselves, or to second guess, or say, ‘‘Oh that must be wrongbecause of that, or something.’

      Gaming referees.

  25. Feb 2016
    1. On the cost rejected papers add to the peer review system (p. 119): "The cost to the academic community of refereeing was estimated by Tenopir and King in 1997 to be $480/article (based on an average time 3–6 hours per article by each of 2–3 referees). At 2004 levels this is approximately $540 per submitted article. Clearly,the percentage ofpapers which are rejected makes a difference to the over- all cost to the journal; in a reasonable quality journal at least 50% of papers will be rejected, while some top journals (e.g. Nature) may reject as many as 90%. Most articles get published somewhere, and as they work their way through the system, being refereed for different journals, they accumulate additional cost; indeed, it couldbe said that a poor (or, at least, inappropriately submitted) article costs the system much more overall than does a good one."

    1. As I have mentioned in previous posts, several platforms have appeared recently that could take on this role of third-party reviewer. I could imagine at least: libreapp.org, peerevaluation.org, pubpeer.com, and publons.com. Pandelis Perakakis mentioned several others as well: http://thomas.arildsen.org/2013/08/01/open-review-of-scientific-literature/comment-page-1/#comment-9.
  26. Jan 2016
    1. Below I list a few advantages and drawbacks of anonymity where I assume that a drawback of anonymous review is an advantage of identified review and vice versa. Drawbacks Reviewers do not get credit for their work. They cannot, for example, reference particular reviews in their CVs as they can with publications. It is relatively “easy” for a reviewer to provide unnecessarily blunt or harsh critique. It is difficult to guess if the reviewer has any conflict of interest with the authors by being, for example, a competing researcher interested in stalling the paper’s publication. Advantages Reviewers do not have to fear “payback” for an unfavourable review that is perceived as unfair by the authors of the work. Some (perhaps especially “high-profile” senior faculty members) reviewers might find it difficult to find the time to provide as thorough a review as they would ideally like to, yet would still like to contribute and can perhaps provide valuable experienced insight. They can do so without putting their reputation on the line.
    1. With most journals, if I submit a paper that is rejected, that information is private and I can re-submit elsewhere. In open review, with a negative review one can publicly lose face as well as lose the possibility of re-submitting the paper. Won’t this be a significant disincentive to submit? This is precisely what we are trying to change. Currently, scientists can submit a paper numerous times, receive numerous negative reviews and ultimately publish their paper somewhere else after having “passed” peer review. If scientists prefer this system then science is in a dangerous place. By choosing this model, we as scientists are basically saying we prefer nice neat stories that no one will criticize. This is silly though because science, more often than not, is not neat and perfect. The Winnower believes that transparency in publishing is of the utmost importance. Going from a closed anonymous system to an open system will be hard for many scientists but I believe that it is the right thing to do if we care about the truth.
    2. PLOS Labs is working on establishing structured reviews and we have talked with them about this.
    3. It should be noted that papers will always be open for review so that a paper can accumulate reviews throughout its lifetime.
  27. Dec 2015
    1. We believe that openness and transparency are core values of science. For a long time, technological obstacles existed preventing transparency from being the norm. With the advent of the internet, however, these obstacles have largely disappeared. The promise of open research can finally be realized, but this will require a cultural change in science. The power to create that change lies in the peer-review process.

      We suggest that beginning January 1, 2017, reviewers make open practices a pre-condition for more comprehensive review. This is already in reviewers’ power; to drive the change, all that is needed is for reviewers to collectively agree that the time for change has come.

  28. Sep 2015
    1. It would avoid any possible awkwardness on either side, and if the discussion were constructive, then both author and commenter would profit from the outcome.

      What's not to like?

      Oh.

  29. Jul 2015
    1. Put simply, various stakeholders seem to have different perspectives on how research assessment works currently and how it should work in the future. In order to move forward, we must first identify and then address a number of misunderstandings.

      This might be of interest, a project that we've put together for the Scholarly Communication Institute this year titled "The Qualities of Quality – Validating and justifying digital scholarship beyond traditional values frameworks": http://trianglesci.org/2015/05/15/the-qualities-of-quality/

    1. Whether or not you take a constructivist view of education, feedback on performance is inevitably seen as a crucial component of the process. However, experience shows that students (and academic staff) often struggle with feedback, which all too often fails to translate into feed-forward actions leading to educational gains. Problems get worse as student cohort sizes increase. By building on the well-established principle of separating marks from feedback and by using a social network approach to amplify peer discussion of assessed tasks, this paper describes an efficient system for interactive student feedback. Although the majority of students remain passive recipients in this system, they are still exposed to deeper reflection on assessed tasks than in traditional one-to-one feedback processes.
  30. May 2015
    1. Broockman has ideas about how to reform things. He thinks so-called “post-publication peer review” — a system which, to oversimplify a bit, makes it easier to evaluate the strength of previously published findings — has promise.
    1. Author and peer reviewer anonymity haven’t been shown to have an overall benefit, and they may cause harm. Part of the potential for harm is if journals act as though it’s a sufficiently effective mechanism to prevent bias.
    2. Peer reviewers were more likely to substantiate the points they made (9, 14, 16, 17) when they knew they would be named. They were especially likely to provide extra substantiation if they were recommending an article be rejected, and they knew their report would be published if the article was accepted anyway (9, 15).
  31. Feb 2014
    1. Alternatively, Daphne Koller and Andrew Ng who are the founders of Coursera, a Stanford MOOC startup, have decided to use peer evaluation to assess writing. Koller and Ng (2012) specifically used the term “calibrated peer review” to refer to a method of peer review distinct from an application developed by UCLA with National Science Foundation funding called Calibrated Peer Review™ (CPR). For Koller and Ng, “calibrated peer review” is a specific form of peer review in which students are trained on a particular scoring rubric for an assignment using practice essays before they begin the peer review process.
  32. Jan 2014
    1. This suggests that peer production will thrive where projects have three characteristi cs

      If thriving is a metric (is it measurable? too subjective?) of success then the 3 characteristics it must have are:

      • modularity: divisible into components
      • granularity: fine-grained modularity
      • integrability: low-cost integration of contributions

      I don't dispute that these characteristics are needed, but they are too general to be helpful, so I propose that we look at these three characteristics through the lens of the type of contributor we are seeking to motivate.

      How do these characteristics inform what we should focus on to remove barriers to collaboration for each of these contributor-types?

      Below I've made up a rough list of lenses. Maybe you have links or references that have already made these classifications better than I have... if so, share them!

      Roughly here are the classifications of the types of relationships to open source projects that I commonly see:

      • core developers: either hired by a company, foundation, or some entity to work on the project. These people care most about integrability.

      • ecosystem contributors: someone either self-motivated or who receives a reward via some mechanism outside the institution that funds the core developers (e.g. reputation, portfolio for future job prospects, tools and platforms that support a consulting business, etc). These people care most about modularity.

      • feature-driven contributors: The project is useful out-of-the-box for these people and rather than build their own tool from scratch they see that it is possible for the tool to work they way they want by merely contributing code or at least a feature-request based on their idea. These people care most about granularity.

      The above lenses fit the characteristics outlined in the article, but below are other contributor-types that don't directly care about these characteristics.

      • the funder: a company, foundation, crowd, or some other funding body that directly funds the core developers to work on the project for hire.

      • consumer contributors: This class of people might not even be aware that they are contributors, but simply using the project returns direct benefits through logs and other instrumented uses of the tool to generate data that can be used to improve the project.

      • knowledge-driven contributors: These contributors are most likely closest to the ecosystem contributors, maybe even a sub-species of those, that contribute to documentation and learning the system; they may be less-skilled at coding, but still serve a valuable part of the community even if they are not committing to the core code base.

      • failure-driven contributors: A primary source of bug reports and may also be any one of the other lenses.

      What other lenses might be useful to look through? What characteristics are we missing? How can we reduce barriers to contribution for each of these contributor types?

      I feel that there are plenty of motivations... but what barriers exist and what motivations are sufficient for enough people to be willing to surmount those barriers? I think it may be easier to focus on the barriers to make contributing less painful for the already-convinced, than to think about the motivators for those needing to be convinced-- I think the consumer contributors are some of the very best suited to convince the unconvinced; our job should be to remove the barriers for people at each stage of community we are trying to build.

      A note to the awesome folks at Hypothes.is who are reading our consumer contributions... given the current state of the hypothes.is project, what class of contributors are you most in need of?