22 Matching Annotations
  1. Aug 2022
    1. Weak sequencing reduces to a parallel merge when the operands are on different sets of participants. Weak sequencing reduces to strict sequencing when the operands work on the same participant.

      weak sequencing - reduces to: - or: - parallel (simultaneous) - strict sequencing

    1. I recommend using the term “parallel” when the simultaneous execution is assured or expected, and to use the term “concurrent” when it is uncertain or irrelevant if simultaneous execution will be employed.
    2. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.
    3. A system is said to be concurrent if it can support two or more actions in progress at the same time. A system is said to be parallel if it can support two or more actions executing simultaneously.
    4. Parallelism means that an application splits its tasks up into smaller subtasks which can be processed in parallel, for instance on multiple CPUs at the exact same time.
    5. Concurrency means executing multiple tasks at the same time but not necessarily simultaneously.
  2. Nov 2021
    1. It partitions optimizer state, gradients and parameters across multiple data parallel processes via a dynamic communication schedule to minimize the communication volume.

      ZeRO-DP 的原理是什么?

    2. Asynchronous parallel (ASP): Every GPU worker processes the data asynchronously, no waiting or stalling. However, it can easily lead to stale weights being used and thus lower the statistical learning efficiency. Even though it increases the computation time, it may not speed up training time to convergence.

      ASP 是什么以及其优缺点?

    3. Bulk synchronous parallels (BSP): Workers sync data at the end of every minibatch. It prevents model weights staleness and good learning efficiency but each machine has to halt and wait for others to send gradients.

      BSP 是什么以及其优缺点?

  3. Aug 2021
  4. Mar 2021
  5. Feb 2021
    1. Rather than implement features you might need, you implement only the features you definitely need, but in a way that accommodates change. If you don't have this flexibility, parallel development simply isn't possible.
    2. At the core of parallel development, however, is the notion of flexibility. You have to write your code in such a way that you can incorporate newly discovered requirements into the existing code as painlessly as possible.
  6. Dec 2020
    1. The problem with using a custom syntax like {#slot} is that we lose the parallel that already exists with native (custom elements) <slot />, and where slot="name", like Svelte currently, also has to be assigned to an element (e.g. <div slot="name">).
  7. Sep 2018
  8. instructure-uploads.s3.amazonaws.com instructure-uploads.s3.amazonaws.com
    1. oneconsistinginthementionofactualpastfacts,theotherintheinventionoffactsbythespeaker.()fthelatter.again,therearetwovan'eties,theillustrativeparalle

      Douglass draws an illustrative parallel to facts of the past when he references the decline of Babylon and uses this parallel to invoke empathy by demonstrating that the celebratory moment of the Fourth of July is built on the suffering of many. Douglass says, “Fellow-citizens; above your national, tumultuous joy, I hear the mournful wail of millions! whose chains, heavy and grievous yesterday, are, to-day, rendered more intolerable by the jubilee shouts that reach them.” Although, some may dispute that the Biblical reference is an actual event (that is an entirely different discussion), for Douglass this parallel is effective because it also appeals to the religious beliefs of the free Americans. It demonstrates the hypocrisy of the actions by the very white men who have invited him to speak to them. On one hand, the freemen appear to be inclusive of slaves because they invited Douglass to speak; on the other hand, they must not forget that this extraordinary jubilee does not include the speaker himself. This inductive technique must lead the freemen to reevaluate their own beliefs and should lead to recognition of how they are a shame before all men and before God.

  9. Sep 2017
    1. convenient & proper part of the State for the University of Virginia.

      While I'm not exactly surprised by the general modern assininity of historic perspectives, it is rather striking to me how parallel this particular phrase is to one in the Constitution, i.e., the necessary and proper clause, or the elastic clause. It makes me wonder whether or not they used this as an excuse to establish whatever they pleased on site. Although eminent domain did not exist in the US officially at the time, and Virginia had a tendency to be ever so slightly anarchistic towards the wealthy, this does pose the question of how the poorer folk in the area got off with the changes, as well as how the university's establishment served as a political tool.

  10. Mar 2017
  11. Sep 2015
    1. Architects continue to be fascinated with finding and describing parallels between symbolic structures and architectural forms.

      As mentioned by someone else in a previous comment, the red archway on Whittier College's campus is supposedly a "symbolic structure" and its architectural characteristics offer an interesting representation of something that looks flexible, but is in fact extremely rigid. What could be relatable to this?