Some may not realize it yet, but the shift in technology represented by ChatGPT is just another small evolution in the chain of predictive text with the realms of information theory and corpus linguistics.

Claude Shannon's work along with Warren Weaver's introduction in The Mathematical Theory of Communication (1948), shows some of the predictive structure of written communication. This is potentially better underlined for the non-mathematician in John R. Pierce's book An Introduction to Information Theory: Symbols, Signals and Noise (1961) in which discusses how one can do a basic analysis of written English to discover that "e" is the most prolific letter or to predict which letters are more likely to come after other letters. The mathematical structures have interesting consequences like the fact that crossword puzzles are only possible because of the repetitive nature of the English language or that one can use the editor's notation "TK" (usually meaning facts or date To Come) in writing their papers to make it easy to find missing information prior to publication because the statistical existence of the letter combination T followed by K is exceptionally rare and the only appearances of it in long documents are almost assuredly areas which need to be double checked for data or accuracy.

Cell phone manufacturers took advantage of the lower levels of this mathematical predictability to create T9 predictive text in early mobile phone technology. This functionality is still used in current cell phones to help speed up our texting abilities. The difference between then and now is that almost everyone takes the predictive magic for granted.

As anyone with "fat fingers" can attest, your phone doesn't always type out exactly what you mean which can result in autocorrect mistakes (see: DYAC (Damn You AutoCorrect)) of varying levels of frustration or hilarity. This means that when texting, one needs to carefully double check their work before sending their text or social media posts or risk sending their messages to Grand Master Flash instead of Grandma.

The evolution in technology effected by larger amounts of storage, faster processing speeds, and more text to study means that we've gone beyond the level of predicting a single word or two ahead of what you intend to text, but now we're predicting whole sentences and even paragraphs which make sense within a context. ChatGPT means that one can generate whole sections of text which will likely make some sense.

Sadly, as we know from our T9 experience, this massive jump in predictability doesn't mean that ChatGPT or other predictive artificial intelligence tools are "magically" correct! In fact, quite often they're wrong or will predict nonsense, a phenomenon known as AI hallucination. Just as with T9, we need to take even more time and effort to not only spell check the outputs from the machine, but now we may need to check for the appropriateness of style as well as factual substance!

The bigger near-term problem is one of human understanding and human communication. While the machine may appear to magically communicate (often on our behalf if we're publishing it's words under our names), is it relaying actual meaning? Is the other person reading these words understanding what was meant to have been communicated? Do the words create knowledge? Insight?

We need to recall that Claude Shannon specifically carved semantics and meaning out of the picture in the second paragraph of his seminal paper:

Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.

So far ChatGPT seems to be accomplishing magic by solving a small part of an engineering problem by being able to explore the adjacent possible. It is far from solving the human semantic problem much less the un-adjacent possibilities (potentially representing wisdom or insight), and we need to take care to be aware of that portion of the unsolved problem. Generative AIs are also just choosing weighted probabilities and spitting out something which is prone to seem possible, but they're not optimizing for which of many potential probabilities is the "best" or the "correct" one. For that, we still need our humanity and faculties for decision making.

Shannon, Claude E. A Mathematical Theory of Communication. Bell System Technical Journal, 1948.

Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. University of Illinois Press, 1949.

Pierce, John Robinson. An Introduction to Information Theory: Symbols, Signals and Noise. Second, Revised. Dover Books on Mathematics. 1961. Reprint, Mineola, N.Y: Dover Publications, Inc., 1980. https://www.amazon.com/Introduction-Information-Theory-Symbols-Mathematics/dp/0486240614.

Shannon, Claude Elwood. “The Bandwagon.” IEEE Transactions on Information Theory 2, no. 1 (March 1956): 3. https://doi.org/10.1109/TIT.1956.1056774.

We may also need to explore The Bandwagon, an early effect which Shannon noticed and commented upon. Everyone seems to be piling on the AI bandwagon right now...

Thomas Kuhn's book The Structure of Scientific Revolutions was directed to scientific thought over time, but he was aware of it potentially being applied, potentially improperly, to other areas. As a result, he narrowed down his definitions and made his assumptions more explicit.

This sort of misapplication can be seen in Social Darwinism, the uncertainty principle, relativity, and memes.

It also happened with Claude Shannon's information theory which resulted in his publication of The Bandwagon (IEEE, 1956).

https://www.youtube.com/watch?v=nS0luYZd4fs

On the Jokerzettel

9/8j Im Zettelkasten ist ein Zettel, der das Argument enthält, das die Behauptungen auf allen anderen Zetteln widerlegt.

Aber dieser Zettel verschwindet, sobald man den Zettelkasten aufzieht.

D.h. er nimmt eine andere Nummer an, verstellt sich und ist dann nicht zu finden.

Ein Joker.

—Niklas Luhmann, ZK II: Zettel 9/8j

Translation:

9/8j In the slip box is a slip containing the argument that refutes the claims on all the other slips. But this slip disappears as soon as you open the slip box. That is, he assumes a different number, disguises himself and then cannot be found. A joker.

Many have asked about the meaning of this jokerzettel over the past several years. Here's my slightly extended interpretation, based on my own practice with thousands of cards, about what Luhmann meant:

Imagine you've spent your life making and collecting notes and ideas and placing them lovingly on index cards. You've made tens of thousands and they're a major part of your daily workflow and support your life's work. They define you and how you think. You agree with Friedrich Nietzsche's concession to Heinrich Köselitz that “You are right — our writing tools take part in the forming of our thoughts.” Your time is alive with McLuhan's idea that "The medium is the message." or in which his friend John Culkin said, "We shape our tools and thereafter they shape us."

Eventually you're going to worry about accidentally throwing your cards away, people stealing or copying them, fires (oh! the fires), floods, or other natural disasters. You don't have the ability to do digital back ups yet. You ask yourself, can I truly trust my spouse not to destroy them?,What about accidents like dropping them all over the floor and needing to reorganize them or worse, the ghost in the machine should rear its head?

You'll fear the worst, but the worst only grows logarithmically in proportion to your collection.

Eventually you pass on opportunities elsewhere because you're worried about moving your ever-growing collection. What if the war should obliterate your work? Maybe you should take them into the war with you, because you can't bear to be apart?

If you grow up at a time when Schrodinger's cat is in the zeitgeist, you're definitely going to have nightmares that what's written on your cards could horrifyingly change every time you look at them. Worse, knowing about the Heisenberg Uncertainly Principle, you're deathly afraid that there might be cards, like electrons, which are always changing position in ways you'll never be able to know or predict.

As a systems theorist, you view your own note taking system as a input/output machine. Then you see Claude Shannon's "useless machine" (based on an idea of Marvin Minsky) whose only function is to switch itself off. You become horrified with the idea that the knowledge machine you've painstakingly built and have documented the ways it acts as an independent thought partner may somehow become self-aware and shut itself off!?!

https://www.youtube.com/watch?v=gNa9v8Z7Rac

And worst of all, on top of all this, all your hard work, effort, and untold hours of sweat creating thousands of cards will be wiped away by a potential unknowable single bit of information on a lone, malicious card and your only recourse is suicide, the unfortunate victim of dataism.

Of course, if you somehow manage to overcome the hurdle of suicidal thoughts, and your collection keeps growing without bound, then you're sure to die in a torrential whirlwind avalanche of information and cards, literally done in by information overload.

But, not wishing to admit any of this, much less all of this, you imagine a simple trickster, a joker, something silly. You write it down on yet another card and you file it away into the box, linked only to the card in front of it, the end of a short line of cards with nothing following it, because what could follow it? Put it out of your mind and hope your fears disappear away with it, lost in your box like the jokerzettel you imagined. You do this with a self-assured confidence that this way of making sense of the world works well for you, and you settle back into the methodical work of reading and writing, intent on making your next thousands of cards.

Hello u/sscheper,

Let me start by thanking you for introducing me to Zettelkasten. I have been writing notes for a week now and it's great that I'm able to retain more info and relate pieces of knowledge better through this method.

I recently came to notice that there is redundancy in my index entries.

I have two entries for Number Line. I have two branches in my Math category that deals with arithmetic, and so far I have "Addition" and "Subtraction". In those two branches I talk about visualizing ways of doing that, and both of those make use of and underline the term Number Line. So now the two entries in my index are "Number Line (Under Addition)" and "Number Line (Under Subtraction)". In those notes I elaborate how exactly each operation is done on a number line and the insights that can be derived from it. If this continues, I will have Number Line entries for "Multiplication" and "Division". I will also have to point to these entries if I want to link a main note for "Number Line".

Is this alright? Am I underlining appropriately? When do I not underline keyterms? I know that I do these to increase my chances of relating to those notes when I get to reach the concept of Number Lines as I go through the index but I feel like I'm overdoing it, and it's probably bloating it.

I get "Communication (under Info. Theory): '4212/1'" in the beginning because that is one aspect of Communication itself. But for something like the number line, it's very closely associated with arithmetic operations, and maybe I need to rethink how I populate my index.

Presuming, since you're here, that you're creating a more Luhmann-esque inspired zettelkasten as opposed to the commonplace book (and usually more heavily indexed) inspired version, here are some things to think about:<br /> - Aren't your various versions of number line card behind each other or at least very near each other within your system to begin with? (And if not, why not?) If they are, then you can get away with indexing only one and know that the others will automatically be nearby in the tree. <br /> - Rather than indexing each, why not cross-index the cards themselves (if they happen to be far away from each other) so that the link to Number Line (Subtraction) appears on Number Line (Addition) and vice-versa? As long as you can find one, you'll be able to find them all, if necessary.

If you look at Luhmann's online example index, you'll see that each index term only has one or two cross references, in part because future/new ideas close to the first one will naturally be installed close to the first instance. You won't find thousands of index entries in his system for things like "sociology" or "systems theory" because there would be so many that the index term would be useless. Instead, over time, he built huge blocks of cards on these topics and was thus able to focus more on the narrow/niche topics, which is usually where you're going to be doing most of your direct (and interesting) work.

Your case sounds, and I see it with many, is that your thinking process is going from the bottom up, but that you're attempting to wedge it into a top down process and create an artificial hierarchy based on it. Resist this urge. Approaching things after-the-fact, we might place information theory as a sub-category of mathematics with overlaps in physics, engineering, computer science, and even the humanities in areas like sociology, psychology, and anthropology, but where you put your work on it may depend on your approach. If you're a physicist, you'll center it within your physics work and then branch out from there. You'd then have some of the psychology related parts of information theory and communications branching off of your physics work, but who cares if it's there and not in a dramatically separate section with the top level labeled humanities? It's all interdisciplinary anyway, so don't worry and place things closest in your system to where you think they fit for you and your work. If you had five different people studying information theory who were respectively a physicist, a mathematician, a computer scientist, an engineer, and an anthropologist, they could ostensibly have all the same material on their cards, but the branching structures and locations of them all would be dramatically different and unique, if nothing else based on the time ordered way in which they came across all the distinct pieces. This is fine. You're building this for yourself, not for a mass public that will be using the Dewey Decimal System to track it all down—researchers and librarians can do that on behalf of your estate. (Of course, if you're a musician, it bears noting that you'd be totally fine building your information theory section within the area of "bands" as a subsection on "The Bandwagon". 😁)

If you overthink things and attempt to keep them too separate in their own prefigured categorical bins, you might, for example, have "chocolate" filed historically under the Olmec and might have "peanut butter" filed with Marcellus Gilmore Edson under chemistry or pharmacy. If you're a professional pastry chef this could be devastating as it will be much harder for the true "foodie" in your zettelkasten to creatively and more serendipitously link the two together to make peanut butter cups, something which may have otherwise fallen out much more quickly and easily if you'd taken a multi-disciplinary (bottom up) and certainly more natural approach to begin with. (Apologies for the length and potential overreach on your context here, but my two line response expanded because of other lines of thought I've been working on, and it was just easier for me to continue on writing while I had the "muse". Rather than edit it back down, I'll leave it as it may be of potential use to others coming with no context at all. In other words, consider most of this response a selfish one for me and my own slip box than as responsive to the OP.)

What a god-awful reference for Claude Shannon. Obviously he found it in his reading through serendipity and didn't bother chasing down the original quote for publication...

9/8j In the slip box is a slip containing the argument that refutes the claims on all the other slips.

But this slip disappears as soon as you open the slip box.

Ie he assumes a different number, disguises himself and then cannot be found.

A joker.

An example of a jokerzettel.

Link this to the Claude Shannon's useless machine (based on an idea of Marvin Minsky) of a useless machine whose only function is to switch itself off. see also https://en.wikipedia.org/wiki/Useless_machine https://www.youtube.com/watch?v=gNa9v8Z7Rac

This same sort of craze also happened with Claude Shannon's The Mathematical Theory of Information which helped to bolster Weiner's take.

A reasonably succinct summary of Claude Shannon's 1948 paper The Mathematical Theory of Communication. By 1981 it had firmly ensconced itself into the vernacular, and would have done so for Luhmann as much of systems theory grew out of the prior generation's communication theory.

Here they're comparing reading for information and reading for understanding.

How do these two modes relate to Claude Shannon's versions of information (surprise) and semantics (the communication) itself. Are there other pieces which exist which we're not tacitly including here? It feels like there's another piece we're overlooking.

His definition of a Memex is simply a mechanized (or what we would now call digitized) commonplace book, which has a long history in the literature of knowledge management.

I'll note here that he's somehow still stuck on the mechanical engineering idea of mechanized. Despite the fact that he was the advisor to Claude Shannon, father of the digital revolution, he is still thinking in terms of mechanical pipes, levers, and fluids. He literally had Shannon building a computer out of pipes and fluid while he was a student at MIT.

I need to track down this reference and would love to see what Weaver has to say about the matter.

Certainly Weaver would have spoken of this with Claude Shannon (or he'd have read it).

<small><cite class='h-cite via'>ᔥ <span class='p-author h-card'>Cory Doctorow</span> in Pluralistic: 16 Feb 2021 – Pluralistic: Daily links from Cory Doctorow (<time class='dt-published'>02/25/2021 12:20:24</time>)</cite></small>

This is a bit disingenuous as Shannon's body of thought preceded that of Dembski by several decades.

Oops, I think this article is confusing Wiener with Claude Shannon?

I'm curious what year this was, particularly in relation to Claude Shannon's master's thesis in which he applied Boolean algebra to electronics.

Based on their meeting date, it would have to be after 1940. And they published in 1943: https://link.springer.com/article/10.1007%2FBF02478259