This famous paper gives a great review of the DQN algorithm a couple years after it changed everything in Deep RL. It compares six different extensions to DQN for Deep Reinforcement Learning, many of which have now become standard additions to DQN and other Deep RL algorithms. It also combines all of them together to produce the "rainbow" algorithm, which outperformed many other models for a while.

Arxiv paper from 2021 on reinforcement learning in a scenario where your aim is to learn a workable POMDP policy, but you start with a fully observable MDP and adjust it over time towards a POMDP.

Paper that introduced the PPO algorithm. PPO is, in a way, a response to the TRPO algorithm, trying to use the core idea but implement a more efficient and simpler algorithm.

TRPO defines the problem as a straight optimization problem, no learning is actually involved.

Bowen Baker et. al. (Open AI) "Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos" Arkiv, June 2022.

Introduction of VPT : New semi-supervied pre-trained model for sequential decision making on Minecraft. Data are from human video playthroughs but are unlabelled.

Liang, Machado, Talvite, Bowling - AAMAS 2016 "State of the Art Control of Atari Games Using Shallow Reinforcement Learning"

Response paper to DQN showing that well designed Value Function Approximations can also do well at these complex tasks without the use of Deep Learning

A great paper showing how to think differently about the latest advances in Deep RL. All is not always what it seems!

Tom Schaul, John Quan, Ioannis Antonoglou and David Silver. "PRIORITIZED EXPERIENCE REPLAY", ICLR, 2016.

Xu, ICCV, 2019 "Temporal Recurrent Networks for Online Action Detection"

arxiv: https://arxiv.org/abs/1811.07391 hypothesis: https://hyp.is/go?url=https%3A%2F%2Fopenaccess.thecvf.com%2Fcontent_ICCV_2019%2Fpapers%2FXu_Temporal_Recurrent_Networks_for_Online_Action_Detection_ICCV_2019_paper.pdf&group=world

(24:58) In order to perform significantly better, you need to FOCUS your attention on a single thing only. Multitasking won't work, and thinking about different things at once also doesn't work. Set up your environment to foster this insane level of focus.

(21:47) There is a clear relation between the body and the brain and they influence each other, at least in terms of perception with regards to motivation.

(13:36) Perhaps this is also why visual information is so useful for learning and cognition (see GRINDE)... Maybe the visual medium should be used more in instruction instead of primarily auditory lectures (do take into account redundancy and other medium effects from CLT though)

GRINDE mapping: 1. Grouped: grouping knowledge together 2. Reflective: reflective of your (non-linear) thinking 3. Interconnected: making more & distant connections (stronger than the groups) 4. Non-verbal (visuals) 5. Directional: which relations are the strongest, in which order can you sequence them? 6. Emphasise (visually) the most important things (see directional as well)

This same sort of structure could be useful for introducing students to fellow college students and also professionals who eschew a hyper-connected, frenetic, algorithmic, hustle mindset.

A way to normalize digital minimalism and slow productivity

To be processed

The author, Rediscovering Analog, reads a book at least twice, usually. He first reads it mainly for pleasure, just to enjoy it and to see what's in it. During the second time, if applicable, he goes through the book using intellectual (or learning) systems and methodologies to extract value from the book.

The first pass, which the author terms Scouting, is thus namely for enjoyment, but keeping in mind what might be valuable or interesting that will be valuable in the future, basically an unguided open ear. He has a list of scouted books in each section of the Zettelkasten that might be relevant to the section. What he does is have a stack of physical cards there with just the name of the book and the author, without anything else. Then when author proceeds to extract value from the book, he takes the card out and puts it in the respective book. Afterwards throwing this particular card into the trash. It's a form of the Anti-Library.

( Personally, I would include an appropriate reading cost and a level on Adler's hierarchy of books. In addition, I would make sure that my process of orientation, in the Inquiry-Based Learning framework, has been completed before I put it as a book within the Anti-Library. )

This may not be the most efficient for the purpose of acquiring value, but efficiency is not all there is. Enjoyment is a big part of intellectual work as well, as Antonin Sertillanges argues in his book The Intellectual Life: Its spirit, methods, conditions, as well as Mihaly Csikszentmihaliy in his book Flow.

What do they mean by "model" here? If they have retrained on more data, with a slightly different architecture, then the model weights after training must be different.

(14:20-19:00) Dopamine Prediction Error is explained by Andrew Huberman in the following way: When we anticipate something exciting dopamine levels rise and rise, but when we fail it drops below baseline, decreasing motivation and drive immensely, sometimes even causing us to get sad. However, when we succeed, dopamine rises even higher, increasing our drive and motivation significantly... This is the idea that successes build upon each other, and why celebrating the "marginal gains" is a very powerful tool to build momentum and actually make progress. Surprise increases this effect even more: big dopamine hit, when you don't anticipate it.

Social Media algorithms make heavy use of this principle, therefore enslaving its user, in particular infinite scrolling platforms such as TikTok... Your dopamine levels rise as you're looking for that one thing you like, but it drops because you don't always have that one golden nugget. Then it rises once in a while when you find it. This contrast creates an illusion of enjoyment and traps the user in an infinite search of great content, especially when it's shortform. It makes you waste time so effectively. This is related to getting the success mindset of preferring delayed gratification over instant gratification.

It would be useful to reflect and introspect on your dopaminic baseline, and see what actually increases and decreases your dopamine, in addition to whether or not these things help to achieve your ambitions. As a high dopaminic baseline (which means your dopamine circuit is getting used to high hits from things as playing games, watching shortform content, watching porn) decreases your ability to focus for long amounts of time (attention span), and by extent your ability to learn and eventually reach success. Studying and learning can actually be fun, if your dopamine levels are managed properly, meaning you don't often engage in very high-dopamine emitting activities. You want your brain to be used to the low amounts of dopamine that studying gives. A framework to help with this reflection would be Kolb's.

A short-term dopamine reset is to not use the tool or device for about half an hour to an hour (or do NSDR). However, this is not a long-term solution.

Huberman states that doing these 4 things consistently and regularly, as a habit, might seem to take time, therefore decreasing performance. BUT, in reality they increase performance, as these things improve your health, focus, and awareness significantly.

Therefore they are so-called Performance Enablers

The 4 (behavioral) keypoints for great physical and mental as well as cognitive health:

One) (2:00-4:05) View sunlight early in the day. The light needs to reach the eyes--increasing alertness, mood, and focus, through certain receptors. Also increases sleep quality at night, according to Huberman. Ideally five to ten minutes on a clear day, and ten to twenty minutes on an overcast day. No sunglasses, and certainly not through windows and windshields. If no sun is out yet, use artificial bright light. Do this daily.

Two) (4:05-6:10) Do physical exercise each and every day. Doesn't have to be super intense. Huberman recommends zone two cardiovascular exercise. Walking very fast, running, cycling, rowing, swimming are examples. He says to get at least between 150 and 200 minutes of this exercise per week. Some resistance training as well for longevity and wellbeing, increases metabolism as well. Do this at least every other day, according to Huberman. Huberman alternates each day between cardiovascular exercise and resistance training.

Three) (6:20-9:10) People should have access to a rapid de-stress protocol or tools. This should be able to do quickly and instantly, without friction. You can just do one breath for destress. ( Deep long breath through nose, one quick breath in nose to completely fill the longs, and then breathe out through mouth long.)

Four) (9:12-14:00) To have a deliberate rewiring nervous system protocol to use. A thing that can be done is NSDR (Non-Sleep Deep Rest protocol), this is specifically to increase energy.

Ideally the NSDR should be done after each learning session as well to imitate deep sleep (REM) and therefore accelerate neuroplasticity and thus rewire the nervous system; increasing the strength of connections between neurons and therefore increase retention significantly.

NSDR is also a process of autonomity and control, it allows one to find that they are in control of their body and brain. It makes one realize that external factors don't necessarily have influence. According to Huberman, NSDR even replenishes dopamine when it is depleted, making it also suitable for increasing motivation.

CIFAR-10 performance results are overestimates since some of the training data is essentially in the test set.

Test Twitter Two

Learning happens when we add memory to thinking. So, thinking precedes learning, and is fundamental to learning.

note to self: is thinking required for memory?

Test Twitter Annotation

good article on learning

https://www.youtube.com/watch?v=TQXMl4GycD0

• (intro & title) Studying is not the same as learning
• Higher order learning is interweaving information (interconnecting, building knowledge in networks and graphs) [a zettelkasten and a commonplace book stimulate higher order learning]

How to learn the art of memory (quick)

Basically, you must be unique. You can't compete when you learn exactly the same as everyone else. The education system sets up to fail and makes you a modern slave.

Przeglądanie rozdziałów książki przed ich przeczytaniem pozwala ułożyć twe myśli. "You’re creating little neural hooks to hang your thinking on, making it easier to grasp the concepts"

Liang, Machado, Talvite, Bowling - AAMAS 2016 "State of the Art Control of Atari Games Using Shallow Reinforcement Learning"

A great paper showing how to think differently about the latest advances in Deep RL. All is not always what it seems!

LeBlanc, D. G., & Lee, G. (2021). General Deep Reinforcement Learning in NES Games. Canadian AI 2021. Canadian Artificial Intelligence Association (CAIAC). https://doi.org/10.21428/594757db.8472938b

The misattribution of this quote often seen in educational settings likely stems from Richards & Rodgers from 1986.

See also: - https://hypothes.is/a/cKMkaAZQEe6dq0fkeyNabA - https://hypothes.is/a/YWrJKgZPEe6dy2sJU5KcSw

The frequent educational quote "Tell me and I forget, teach me and I remember, involve me and I learn.", often misattributed to Benjamin Franklin, is most attributable to 3rd century Confucian philosopher Kunzi (Xun Kuang or 荀子) who wrote:

Not having heard of it is not as good as having heard of it. Having heard of it is not as good as having seen it. Having seen it is not as good as knowing it. Knowing it is not as good as putting it into practice. Learning arrives at putting it into practice and then stops . . .

The translation of which appears in Xunzi: The Complete Text, Translated by Eric L. Hutton, Chapter 8: The Achievements of the Ru, Quote Page 64, Princeton University Press, Princeton, New Jersey. 2014.

Variations of the sentiment and attributions have appeared frequently thereafter.

Designing new learning technology is not always the best solution without paying attention to its learning environment.

Blog post comparing ASG (Auto Segmentation Criterion - yes, the last letter doesn't match) to CTC (Connectionist Temporal Classification) for aligning speech recognition model outputs with a transcript.

this reminds me of the "whiteboard effect" and the concept of collaborative learning as described by cal newport in his book, deep work.

such dynamics cultivate a culture of fortuitous learning and the exchange of ideas. when another individual is present, it instills a sense of accountability and motivation to dive profoundly into a problem and the gaps of each other's knowledge than we might when woking in solitude.

• Quote worthy
  • "“Protracted immaturity and dependence on paternal care
    • is not an unfortunate byproduct of our evolution
    • but instead a highly adaptive trait of our species,
    • which has enabled human infants to
      • efficiently organize attention to social agents and
      • learn efficiently from social output,”
  • “The evolutionary goal of altricial species is
    • not to become highly competent as quickly as possible
    • but rather to excel at learning over time.”
• Authors
  • Michael Goldstein,
  • Katerina Faust,
  • Samantha Carouso-Peck
  • Mary R. Elson

• the beauty of perceptual immaturity in altricial species is that
• it makes learning easier by reducing the complexity of the world,” the researchers wrote.
• Parents are key to altricial learning, Goldstein said,
  • forming a two-way system of feedback.
• Far from being passive recipients, he said,
  • infants of many species can change the behavior of their parents
  • in ways that actively shape their own developments.

• Title
  • The Origins of Social Knowledge in Altricial Species,
• Journal
  • The Annual Review of Developmental Psychology, - -
• Publication Date
  • Dec, 2021
• Authors
• Michael Goldstein,
• Katerina Faust,
  • Samantha Carouso-Peck and
  • Mary R. Elson

[29] AI - Deep Learning

https://pressbooks.pub/illuminated/

A booklet prepared for teachers that introduces key concepts from the Science of Learning (i.e. cognitive neuroscience). The digital booklet is the result of a European project. Its content have been compiled from continuing professional development workshops for teachers and features evidence-based teaching practices that align with our knowledge of the Science of Learning.

More than just taking notes - Learning exhaust by Nicole van der Hoeven

https://www.youtube.com/watch?v=3L24rKggMX8

Nice framing to broadly define note taking as a form of learning exhaust, but broadly nothing new here for me.

El e-learning proviene de los términos electronic-learning (aprendizaje electrónico o formación en línea), el cual consiste precisamente en llevar a cabo un proceso de aprendizaje utilizando algún tipo de dispositivo electrónico (ordenador, tablet, smartphone), lo cual y gracias al uso de internet, permite una mayor accesibilidad a la información dando lugar a que en un proceso educativo, éste pueda llevarlo el estudiante sin importar el lugar en que se encuentre, siempre y cuando se disponga de los recursos de comunicación, equipos y herramientas tecnológicas necesarias, situación que vino a darle el impulso a la creación de los estudios de carácter virtual a cualquier nivel, aunque en un principio se inició en el nivel superior, sin embargo, `por diferentes situaciones a venido abarcando otros niveles educativos.

El sistema de aprendizaje blended learning o b-larning, debemos entenderlo como una combinación entre el aprendizaje presencial y el aprenizaje en línea. Lo cual aparece como una opción al surgir las diferentes herramientas tecnológicas y se considera el uso de ellas pero sin dejar de aprovechar los aspectos valiosos que contiene el aprendizaje presencial. Es por ello que a esta "mezcla" de modalidades se ha dado por nombrar también como: Sistema híbrido, mixto, intercambiable, etc. Por ello es de entenderse y aceptarse que es válido el que se quiera con ello aprovechar las cualidades y ventajas que nos concede el aprendizaje virtual y todo aquello que también sigue siendo válido y útil de las sesiones presenciales.

Learning as a Network LaaN and Personal Knowledge Network PKN , do these labels give me anything new?

Mohamed Amine Chatti: The LaaN Theory. In: Personalization in Technology Enhanced Learning: A Social Software Perspective. Aachen, Germany: Shaker Verlag, 2010, pp. 19-42. http://mohamedaminechatti.blogspot.de/2013/01/the-laan-theory.html I've followed Chatti's blog in the past I think. Prof. Dr. Mohamed Amine Chatti is professor of computer science and head of the Social Computing Group in the Department of Computer Science and Applied Cognitive Science at the University of Duisburg-Essen. (did his PhD at RWTH in 2010, which is presumably how I came across him, through Ralf Klamma)

MACHINE LEARNING FOR STOCK PRICES FORECASTING

Notes can be used for different purposes.

• productivity
• Knowledge
  • basic sense-making
  • knowledge construction and dispersion

The broad distinction is between productivity goals and knowledge. (Is there a broad range I'm missing here within the traditions?) You can take notes about projects that need to be taken care of, lists of things to do, reminders of what needs to be done. These all fall within productivity and doing and checking them off a list will help one get to a different place or location and this can be an excellent thing, particularly when the project was consciously decided upon and is a worthy goal.

Notes for knowledge sake can be far more elusive for people. The value here generally comes with far more planning and foresight towards a particular goal. Are you writing a newsletter, article, book, or making a video or performance of some sort (play, movie, music, etc.)? Collecting small pieces of these things on a pathway is then important as you build your ideas and a structure toward some finished product.

Often times, before getting to this construction phase, one needs to take notes to be able to scaffold their understanding of a particular topic. Once basically understood some of these notes may be useless and not need to be reviewed, or if they are reviewed, it is for the purpose of ensconcing ideas into long term memory. Once this is finished, then the notes may be broadly useless. (This is why it's simple to "hide them with one's references/literature notes.) Other notes are more seminal towards scaffolding ideas towards larger projects for summarization and dissemination to other audiences. If you're researching a topic, a fair number of your notes will be used to help you understand the basics while others will help you to compare/contrast and analyze. Notes you make built on these will help you shape new structures and new, original thoughts. (note taking for paradigm shifts). These then can be used (re-used) when you write your article, book, or other creative project.

Devising a prompt (AKA a question) is the key to ChatGPT. I am still uncertain what a good question is in AI's "mind". It might be something "way strange" and "un-questionly".

depends on how you look at it: "normal"

A new import is the _LRScheduler which we will use to implement our learning rate finder.

We will also show how to initialize the weights of our neural network and how to find a suitable learning rate using a modified version of the learning rate finder.

Vicky Zhao indirectly frames the answer for "why have a zettelkasten?", especially for learning, as overcoming the "illusion of competence" which is closely related to the mere-exposure effect and the Dunning–Kruger effect.

This is quite important. Success is achieving the goal. If the goal is unattainable, then so is success.This is how we set students up for failure.

This may benefit from adoption of SOLO and simplification of the rubric model.

Resembles SOLO, though the rubric is wordy and includes a lot of subjective descriptors. Each level is distinct and adds a level of complexity to the previous level. "Expert" clearly aligns with SOLO extended/abstract level. Levels are not clearly linked to levels of cognitive function (uni / multi structural / relational, etc.)

The classification is anything but indifferent. The manner of shelving the books is meant to impart certain suggestions to the reader, who, looking on the shelves for one book, is attracted by the kindred ones next to it, glances at the sections above and below, and finds himself involved in a new trend of thought which may lend to additional interests to the one he was pursuing.<br /> —Claudia Wedepohl on the design of Warburg's library, [00:44:19] in Aby Warburg: Metamorphosis and Memory

Provides a similar sort of description of the push towards serendipity and discovery found in one's zettelkasten as well as that in Melvil Dewey's library classification and arrangements.

For the intellectual worker, an Antinet can be used to keep track of such questions and the thought-lines corresponding to these questions.

As an educator, be courageous, lead by example. Start by asking questions out loud, not only those you wish students to answer, but also those you genuinely don't know, and wish to research together with your students.

Asking questions is how we learn. Do not avoid it for the sake of not looking stupid. That is stupid. Inquiry-Based Learning.

As Confucius said: "The one who asks a question is a fool for a minute, the one who doesn't ask is a fool for life."

Myths about self-regulated learning to be aware of.

Too many zettelkasten aspirants only want the presupposed "rules" for keeping one or are interested in imitating one or another examples. Few have interest in the actual day to day practice and these are often the most adept. Of course the downside of learning some of the pieces online leaves the learner with some (often broken) subset of rules and one or two examples (often only theoretical) and then wonder why their actual practice is left so wanting.

link to https://hypothes.is/a/ZeZEgNm8Ee2woUds5QzgOw

Going back at least as far as the rhetoric of Aristotle, Cicero, and Quintilian, we recognize that while sets of rules can be helpful to the student, these must also be paired with ample imitation and practice.

Bowen Baker et. al. (Open AI) "Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos" Arkiv, June 2022.

New supervised pre-trained model for sequential decision making on Minecraft. Data are from human video playthroughs but are unlabelled.

reinforcement-learning foundation-models pretrained-models proj-minerl minecraft

Interleaving

What would a list of prompts for teaching creatively look like if it were created using Brian Eno's Oblique Strategies as a model?

Inspired by Sophia Rahming (~09:48)

As a teacher of English to secondary school students, and as an online doctoral student, I am excited to explore and possibly integrate Hypothesis into my work. I love research and everything involved with it. Thank you to the creators of this tool --

Definition - socially based individual learning - an individual learns by interacting with the embedded environment - but the environment is itself biased - so that certain learning outcomes - are more easily learned - than they would otherwise be

moss sponging by chmpanzees - is a phenomena observed by researchers - in which the distinction between<br /> - individual and - collective learning - is fuzzy - Sponging is a technique of wild chimpanzees - in which they use chewed up plant material - as a sponge to soak up water - One individual wild chimpanzee - named by the researchers as KW - picked up a discarded sponge used by another wild chimpanzee - which happened to have moss in it - and so developed a sponge for water specifically from moss - KW did not learn it socially from another chimpanzee - yet if it weren't for - the behavior of other chimpanzees in the group - cultural artefacts they left behind - niche construction that resulted to changes in the environment - the individual learning of KW would never have produced moss sponging

• This primate example demonstrates an ambiguity between individual and social learning.
• The problem with this way of defining things exclusively as either
  • individual or
  • social
• is that we ignore the fact that,
  • even when acting in a manner
  • that appears to involve no direct interaction with other creatures,
• organisms nonetheless develop and learn in environments
  • that have been affected by the prior actions of their conspecifics (and not just their conspecifics).
• This is precisely the sort of phenomenon
  • stressed by the proponents of the niche-construction approach to evolution,
  • and it is also stressed by developmental systems theorists [40,41].
• Organisms grow in environments that have been constructed
  • by the actions of previous generations:
    • in that way, what an organism learns
    • can be profoundly affected and enhanced
    • by the collective activities of individuals it may never meet.
• In other words, we should not assume
  • that there is any good distinction
  • between
    • individual learning and
    • what we might call ‘social transmission’.
• The latter can be achieved via the former.

I think this would be the best approach to help children that are slower than "normal" children. I stated earlier that the best way to help children learn is to see how they learn such as being hands on, visual, or auditory learning. Everyone doesn't learn stuff at the exact same time, it may take some people longer to understand the concept than other people. Everyone learns differently. This is important to the history of psychology because it helps us understand that children who are slower than other children can develop intelligence at their own speed. I think another reason this is important to the history of psychology is because we now understand the grade of intelligence is different for everyone, but people can build intelligence at their own progressive rate and that we can't force someone who is slower at learning to be at normal speed.

Center for Innovations in Teaching and Learning

// - this is the mechanism by which ritual practice can bring about interpretive shift unconsciously -because bodily movements have a priming effect - Lakoff points to the concept of "biconceptualism, a compartmentalism in our physical brain

// in other words, - interpretive shift is unconscious and brought about through practice. It is a shift in ontology, habitus and many things happening at once and is also Polyani's tacit learning

//Summary of Haluza-Delay and Polylani's conception of Tacit Knowledge - Haluza-Delay’s description of informal and incidental learning sounds much like Michael Polanyi’s (1974) “practical knowledge.” - Haluza-Delay discusses it as tacit learning, a term Polanyi introduced. - From Polanyi’s description, much of tacit learning is: - initially conscious, - but subsides into subsidiary awareness - People learn values in this fashion, but core values are picked up through imitation without conscious awareness.

Machine learning like money laundering for bias

https://chat.d-id.com/ DID에서 나온 새로운 서비스 음성으로 채팅을 하고 AI와 얼굴을 보며 대화 20번 무료 사용 가능

There's a danger to using digital tools to help with Higher Order Thinking; namely, it offloads precious cognitive load, optimized intrinsic load, which is used to build schemas and structural knowledge which is essential for mastery. Another danger is that digital tools often make falling for the collector's fallacy easier, meaning that you horde and horde information, which makes you think you have knowledge, while in fact, you simply have (maybe related) information, not mastery. The analog way prevents this, as it forces you to carefully evaluate the value of an idea and decide whether or not it's worth it to spend time on writing it and integrating it into a line of thought. Evaluation/Analysis is forced in an analog networked thinking tool, which is a form of Higher Order Learning/Thinking, as they are in the higher orders of Bloom's Taxonomy/Hierarchy.

This is also true for AI. Always carefully evaluate whether or not a tool is worth using, like a farmer. (Deep Work, Cal Newport).

Instead, use a tool like mindmapping, the GRINDE way, which is digital, for learning... Or the Antinet Zettelkasten by Scott Scheper, which is analog, for research.

The power of an Antinet Zettelkasten. Use this principle both in research and learning.

Another social factor influencing human behaviour. Beware of such factors when it comes to self-improvement and learning.

Seems similar to Communicating with an Antinet Zettelkasten.

Need to incorporate this definition into my professional practices

They demonstrate the model on a "minecraft-like" domain (introduced earlier by someone else) where there are resources in the world and the agent has tasks.

Kudos! A rare find these days.

Julian dives into https://medium.com/@ElizAyer/meetings-are-the-work-9e429dde6aa3 on when meetings are productive work (vgl [[Netwerkleren Connectivism 20100421081941]]

No new physics and no new mathematics was discovered by the AI. The AI did however deduce something from the existing math and physics, that no one else had yet seen. Skynet is not coming for us yet.

We can watch adults model things, but we need people to teach us the nuance and context of those behaviors.

There's some truth and falsity here...

I've always disliked the idea of "work" involved in "making" notes and "processing" them. Framing zettelkasten and knowledge creation in terms of capitalism is a painful mistake.

the quote is from https://blay.se/2015-06-21-living-with-a-zettelkasten.html

The inflection point of when something is learned comes in demonstration? Or in spontaneous performance of the behavior?

Is this in conflict with the statement earlier of learning with no demonstration of new behaviors?

Retrieval practice is a method which can be used to reinforce retention.

The MDP does not change, it's dynamics are the same, with or without the RM, as they are with or without a standard reward model. Additionally, the rewards from the RM can be non-Markovian with respect to the MDP because they inherently have a kind of memory or where you've been, limited to the agents "movement" (almost "in it's mind") about where it is along the goals for this task.

So by specifying a reward machine you are augmenting the state space of the MDP with higher level goals/subgoals/concepts that provide structure about what is good and what isn't.

Fascinating idea, why not? Why are we hiding the reward from the agent really?

[Icarte, JAIR, 2022] "Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning"

[Icarte, PMLR, 2018] "Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning"