23 Matching Annotations
  1. Dec 2023
    1. What's the difference between concurrency and parallelism?

      concurrent process performs multiple tasks at the same time whether they're being diverted total attention or not, a parallel process is physically performing multiple tasks all at the same time.

    2. What is parallelism?

      Parallelism is very-much related to concurrency. In fact, parallelism is a subset of concurrency: whereas a concurrent process performs multiple tasks at the same time whether they're being diverted total attention or not, a parallel process is physically performing multiple tasks all at the same time.

    3. There are many reasons your applications can be slow. Sometimes this is due to poor algorithmic design or the wrong choice of data structure. Sometimes, however, it's due to forces outside of our control, such as hardware constraints or the quirks of networking.

      That's where concurrency and parallelism fit in. They allow your programs to do multiple things at once, either at the same time or by wasting the least possible time waiting on busy tasks.

    1. Running the code in a subprocess is much slower than running a thread, not because the computation is slower, but because of the overhead of copying and (de)serializing the data. So how do you avoid this overhead?

      Reducing the performance hit of copying data between processes:

      Option #1: Just use threads

      Processes have overhead, threads do not. And while it’s true that generic Python code won’t parallelize well when using multiple threads, that’s not necessarily true for your Python code. For example, NumPy releases the GIL for many of its operations, which means you can use multiple CPU cores even with threads.

      ``` # numpy_gil.py import numpy as np from time import time from multiprocessing.pool import ThreadPool

      arr = np.ones((1024, 1024, 1024))

      start = time() for i in range(10): arr.sum() print("Sequential:", time() - start)

      expected = arr.sum()

      start = time() with ThreadPool(4) as pool: result = pool.map(np.sum, [arr] * 10) assert result == [expected] * 10 print("4 threads:", time() - start) ```

      When run, we see that NumPy uses multiple cores just fine when using threads, at least for this operation:

      $ python numpy_gil.py Sequential: 4.253053188323975 4 threads: 1.3854241371154785

      Pandas is built on NumPy, so many numeric operations will likely release the GIL as well. However, anything involving strings, or Python objects in general, will not. So another approach is to use a library like Polars which is designed from the ground-up for parallelism, to the point where you don’t have to think about it at all, it has an internal thread pool.

      Option #2: Live with it

      If you’re stuck with using processes, you might just decide to live with the overhead of pickling. In particular, if you minimize how much data gets passed and forth between processes, and the computation in each process is significant enough, the cost of copying and serializing data might not significantly impact your program’s runtime. Spending a few seconds on pickling doesn’t really matter if your subsequent computation takes 10 minutes.

      Option #3: Write the data to disk

      Instead of passing data directly, you can write the data to disk, and then pass the path to this file: * to the subprocess (as an argument) * to parent process (as the return value of the function running in the worker process).

      The recipient process can then parse the file.

      ``` import pandas as pd import multiprocessing as mp from pathlib import Path from tempfile import mkdtemp from time import time

      def noop(df: pd.DataFrame): # real code would process the dataframe here pass

      def noop_from_path(path: Path): df = pd.read_parquet(path, engine="fastparquet") # real code would process the dataframe here pass

      def main(): df = pd.DataFrame({"column": list(range(10_000_000))})

      with mp.get_context("spawn").Pool(1) as pool:
          # Pass the DataFrame to the worker process
          # directly, via pickling:
          start = time()
          pool.apply(noop, (df,))
          print("Pickling-based:", time() - start)
          # Write the DataFrame to a file, pass the path to
          # the file to the worker process:
          start = time()
          path = Path(mkdtemp()) / "temp.parquet"
              # Run faster by skipping compression:
          pool.apply(noop_from_path, (path,))
          print("Parquet-based:", time() - start)

      if name == "main": main() `` **Option #4:multiprocessing.shared_memory`**

      Because processes sometimes do want to share memory, operating systems typically provide facilities for explicitly creating shared memory between processes. Python wraps this facilities in the multiprocessing.shared_memory module.

      However, unlike threads, where the same memory address space allows trivially sharing Python objects, in this case you’re mostly limited to sharing arrays. And as we’ve seen, NumPy releases the GIL for expensive operations, which means you can just use threads, which is much simpler. Still, in case you ever need it, it’s worth knowing this module exists.

      Note: The module also includes ShareableList, which is a bit like a Python list but limited to int, float, bool, small str and bytes, and None. But this doesn’t help you cheaply share an arbitrary Python object.

      A bad option for Linux: the "fork" context

      You may have noticed we did multiprocessing.get_context("spawn").Pool() to create a process pool. This is because Python has multiple implementations of multiprocessing on some OSes. "spawn" is the only option on Windows, the only non-broken option on macOS, and available on Linux. When using "spawn", a completely new process is created, so you always have to copy data across.

      On Linux, the default is "fork": the new child process has a complete copy of the memory of the parent process at the time of the child process’ creation. This means any objects in the parent (arrays, giant dicts, whatever) that were created before the child process was created, and were stored somewhere helpful like a module, are accessible to the child. Which means you don’t need to pickle/unpickle to access them.

      Sounds useful, right? There’s only one problem: the "fork" context is super-broken, which is why it will stop being the default in Python 3.14.

      Consider the following program:

      ``` import threading import sys from multiprocessing import Process

      def thread1(): for i in range(1000): print("hello", file=sys.stderr)


      def foo(): pass

      Process(target=foo).start() ```

      On my computer, this program consistently deadlocks: it freezes and never exits. Any time you have threads in the parent process, the "fork" context can cause in potential deadlocks, or even corrupted memory, in the child process.

      You might think that you’re fine because you don’t start any threads. But many Python libraries start a thread pool on import, for example NumPy. If you’re using NumPy, Pandas, or any other library that depends on NumPy, you are running a threaded program, and therefore at risk of deadlocks, segfaults, or data corruption when using the "fork" multiprocessing context. For more details see this article on why multiprocessing’s default is broken on Linux.

      You’re just shooting yourself in the foot if you take this approach.

    1. Gunicorn and multiprocessing

      Gunicorn forks a base process into n worker processes, and each worker is managed by Uvicorn (with the asynchronous uvloop). Which means:

      • Each worker is concurrent
      • The worker pool implements parallelism

      This way, we can have the best of both worlds: concurrency (multithreading) and parallelism (multiprocessing).

    1. The typical analogy is this:
      • concurrency is having two lines of customers ordering from a one cashier;
      • parallelism is having two lines of customers ordering from two cashiers.
    2. parallelism

      "doing lots of things at once" (As Rob Pike said)

    1. You can distribute work to a bunch of process workers or thread workers with a few lines of code:

      ```python from concurrent.futures import ThreadPoolExecutor, as_completed

      with ThreadPoolExecutor(max_workers=5) as executor: executor.submit(do_something_blockint) ```

  2. Aug 2023
  3. Jun 2023
  4. Aug 2022
  5. Apr 2022
  6. Jan 2022
    1. He leaves no loose ends, no ambiguities, no extravagances, no extremes.

      Brody creates parallelism here by starting each thing with no and having each word end with an S. This creates a list of ideas.

  7. Jan 2021
    1. cybernetic ecology

      First comes the cybernetic meadow, then the cybernetic forest, and ultimately the cybernetic ecology. Brautigan profiles this expansion in third line of each stanza. i wonder where else we might find correspondences of the sort.

  8. Nov 2020
    1. it had been dark, silent, beautiful very often—oh yes—but mournful somehow

      This gives a hint of how life must have been for Leila living in a country. "Dark" and "silent" allude to her being alone, given that she is an only child, which also accounts that her life must have been quite dull and lonely. Yet, she also mentions that most of her nights are "beautiful," which elucidates that she has been living a good life. Perhaps she lives in a lovely house, and her family owns a nice and vast farm, given that she also comes from the same class of family as the Sheridans.

  9. Jul 2019
    1. In an awful pirate fight

      a dream with a fight,a struggle(parallels the boy's struggle facing lonliness

  10. Jan 2017
    1. Signifies that from you great Rome shall suck Reviving blood, and that great men shall press For tinctures, stains, relics, and cognizance.

      How effective is the use of parallelism?

      For more rhetorical devices, consult Handbook.

  11. Sep 2016
    1. cooperating

      This word is used in almost every paragraph of the piece to further the point of working WITH Mexico instead of AGAINST.

    2. innovations

      oops he did it again

    3. innovation

      This word has been repeated several times in order to play on the average american's idealism of "American Innovation"

    1. We've begun initiatives to cooperate on health and agriculture, education and law enforcement.  We've reached agreements to restore direct flights and mail service.  We've expanded commercial ties, and increased the capacity of Americans to travel and do business in Cuba.

      Obama uses parallelism here to convey that Cubans and Americans are united. He does so through the use of the word "we've" to show that strides toward cooperation have already been made.

    2. Cuba has a one-party system; the United States is a multi-party democracy.  Cuba has a socialist economic model; the United States is an open market.  Cuba has emphasized the role and rights of the state; the United States is founded upon the

      President Obama is using parallelism to display the differences and compare America and Cuba. He is doing this to develop a relationship and promote trust between the two countries.