28 Matching Annotations
  1. Jan 2025
    1. for - Youtube - Digital Drip: The Imperceptible Flows of E-Waste - Filip Vedra - WS23 Planet B UMPRUM - 2025, Jan - adjacency - Deep Humanity - sensory bubble - social norm of producer-consumer split and alienation - spread by Industrial Revolution - hyperobjects - source - Youtube - Digital Drip: The Imperceptible Flows of E-Waste - Filip Vedra - WS23 Planet B UMPRUM - 2025, Jan

      // - comment - An insightful documentary that examines the social norm amplified by the Industrial Revolution, - the producer-consumer split and resulting alienation - Globalization has further exasperated this as global supply chains are hyperobjects which no individual can truly sense the scale of

  2. Oct 2024
  3. May 2024
  4. Apr 2024
  5. Nov 2023
    1. Economies that are heavily reliant on oil and gas revenues face some stark choices and pressures in energy transitions.
      • for: stats - oil and gas - steep drop in revenues of fossil fuel producer economies

      • stats: oil and gas - steep drop in revenues of fossil fuel reliant economies

        • per capita net income from oil and natural gas among producer economies will be 60% lower in 2030 in a 1.5 °C scenario.relative to revenues between 2010 and 2022.
      • question

        • many producer economies are not diversifying into clean energy fast enough to compensate for these steep revenue drops
  6. Oct 2023
  7. Aug 2023
  8. Oct 2021
  9. Aug 2021
  10. Apr 2021
  11. Mar 2021
  12. Oct 2020
  13. May 2020
  14. Jun 2017
    1. By electing a new leader as soon as possible messages may be dropped but we will minimized downtime as any new machine can be leader.

      two scenarios to get the leader back: 1.) Wait to bring the master back online. 2.) Or elect the first node that comes back up. But in this scenario if that replica partition was a bit behind the master then the time from when this replica went down to when the master went down. All that data is Lost.

      SO there is a trade off between availability and consistency. (Durability)

  15. May 2017
    1. The Kafka cluster retains all published records—whether or not they have been consumed—using a configurable retention period. For example, if the retention policy is set to two days, then for the two days after a record is published, it is available for consumption, after which it will be discarded to free up space. Kafka's performance is effectively constant with respect to data size so storing data for a long time is not a problem.

      irrespective of the fact that the consumer has consumed the message that message is kept in kafka for the entire retention policy duration.

      You can have two or more consumer groups: 1 -> real time 2 -> back up consumer group

    2. replication factor N, we will tolerate up to N-1 server failures without losing any records

      Replication Factor means number of nodes/brokers which could go down before we start losing data.

      So if you have a replication factor of 6 for a 11 node cluster, then you will be fault tolerant till 5 nodes go down. After that point you are going to loose data for a particular partition.

    3. Messages sent by a producer to a particular topic partition will be appended in the order they are sent. That is, if a record M1 is sent by the same producer as a record M2, and M1 is sent first, then M1 will have a lower offset than M2 and appear earlier in the log.

      ordering is guaranteed.

    1. the only metadata retained on a per-consumer basis is the offset or position of that consumer in the log. This offset is controlled by the consumer: normally a consumer will advance its offset linearly as it reads records, but, in fact, since the position is controlled by the consumer it can consume records in any order it likes.

      partition offset maintained by kafka. Offset number is maintained so that if the consumer goes down nothing breaks.