Author Mary Burns discusses the key elements of computer adaptive testing (CAT). CAT is defined as assessments that use algorithms to progressively adjust test difficulty based upon learner's correct or incorrect responses. The benefits of CAT include more immediate data and are often more reliable. Types of test items are also covered to illustrate how the test can meet various levels of cognition and measure expertise. An issue related to CAT is the intensive time needed to develop multiple test items at multiple levels of cognition. Rating: 8/10

The Office of Educational Technology website that is featured on Section 4: Measuring of Learning, discusses pedagogical implications of assessment shifts and how technology is part of enhancing assessment. The site places emphasis on using assessment as a real-time measurement of learning for real-life skills. The infographic that displays traditional models of assessment next to next-generation assessments is a quick reference for shifting ideology and practices. Ultimately, increased personalized learning serves the learner more efficiently and promotes increased achievement and mastery. Rating: 8/10

Pscyho-technology ~ Mental Models ~ Mental Frameworks

3

2

1

three definitive models for Universal Health Care

Gagne's nine events of instruction I am including this page for myself because it is a nice reference back to Gagne's nine events and it gives both an example of each of the events as well as a list of four essential principles. It also includes some of his book titles. rating 4/5

Edward Thorndike's three laws of learning. The page does not explain this, but his theories came out in about 1900. His three laws of learning appear to be relevant to our course work. This simple page features black text on a white page. It is brief and it simply describes the three laws of learning. rating 5/5

Jack Phillips and ROI. This page describes the Phillips Return on Investment model. The model as presented here is an alternative to Kirkpatrick's model. There's a bulleted list of the components of the model as well as a nice graphic that briefly describes the levels. There is an explanation about how to apply the model, though I think more information would be needed for real world practice. Rating 4/5

This is Bloom's taxonomy of cognitive objectives. I selected this page because it explains both the old and new versions of the taxonomy. When writing instructional objectives for adult learning and training, one should identify the level of learning in Blooms that is needed. This is not the most attractive presentation but it is one of the more thorough ones. rating 4/5

Human Performance Technology Model This page is an eight page PDF that gives an overview of the human performance technology model. This is a black and white PDF that is simply written and is accessible to the layperson. Authors are prominent writers in the field of performance technology. Rating 5/5

This link is to a three-page PDF that describes Gagne's nine events of instruction, largely in in the form of a graphic. Text is minimized and descriptive text is color coded so it is easy to find underneath the graphic at the top. The layout is simple and easy to follow. A general description of Gagne's work is not part of this page. While this particular presentation does not have personal appeal to me, it is included here due to the quality of the page and because the presentation is more user friendly than most. Rating 4/5

This page is a simply presented list of many learning theories, both popular and less well known. The layout is clean. The pages to which the listed items link are somewhat minimal in nature so this would give a basic tour or overview of the models and would allow viewers to review the names of some of the learning theories. This page does not prioritize learning theories or identify and establish those theories that are the most prominent.

This 69 page PDF offers good advice on writing a variety of types of test questions. It is called "Is this a trick question?" Despite the length of the PDF, it is easy to browse if you are interested in writing a specific type of question. As may be suggested by the length, this resource is more comprehensive than others. Rating 5/5

This page describes a method of teaching designed specifically for adults. The instructional design theory is Keller's "ARCS," which stands for attention, relevance, confidence, and satisfaction--all features that adult learning experiences should be characterized by. The text on this page is readable but the popups and graphics are a bit annoying. rating 3/5

This is a description of the form of backward design referred to as Understanding by Design. In its simplest form, this is a three step process in which instructional designers first specify desired outcomes and acceptable evidence before specifying learning activities. This presentation may be a little boring to read as it is text-heavy and black and white, but those same attributes make it printer friendly. rating 3/5

This is the current model of Perkeep.

First mention: https://youtu.be/PlAU_da_U4s?start=630&end=651

More discussion including a very interesting note on what they call Namespace later: https://youtu.be/PlAU_da_U4s?start=2677&end=2787

Related: http://worrydream.com/ExplorableExplanations/

A comment at the bottom by Barbara H Partee, another panelist alongside Chomsky:

I'd like to see inclusion of a version of the interpretation problem that reflects my own work as a working formal semanticist and is not inherently more probabilistic than the formal 'generation task' (which, by the way, has very little in common with the real-world sentence production task, a task that is probably just as probabilistic as the real-world interpretation task).

This article makes a solid argument for why statistical and probabilistic models are useful, not only for prediction, but also for understanding. Perhaps this is a key point that Noam misses, but the quote narrows the definition to models that approximate "unanalyzed data".

However, it seems clear from this article that the successes of ML models have gone beyond approximating unanalyzed data.

Scientific insight isn't the same as metaphysical questions, in spite of having the same question word. Asking, "Why do epidemics have a peak?" is not the same as asking "Why does life exist?". Actually, that second question can be interested in two different ways, one metaphysically and one physically. The latter interpretation means that "why" is looking for a material cause. So even simple and approximate models can have generalizing value, such as the Schelling Segregation model. There is difference between models to predict and models to explain, and both have value. As later mentioned in this document, theory and data are two feet and both are needed for each other.

This page discusses different types of models

• statistical models
• probabilistic models
• trained models

and explores the interaction between prediction and insight.

Wasn't one of Chomsky's ideas that humans are uniquely suited to language? The counter-perspective espoused here appears to be that language emerges, and that humans are only distinguished by the magnitude of their capacity for language; other species probably have proto-language, and there is likely a smooth transition from one to the other. In fact, there isn't a "one" nor an "other" in a true qualitative sense.

So what if we discover something about the human that appears to be required for our language? Does this, then, lead us to knowledge of how human language is qualitatively different from other languages?

Can probabilistic models account for qualitative differences? If a very low, but not 0, probability is assigned to a given event that we know is impossible from our theory-based view, that doesn't make our probabilistic model useless. "All models are wrong, some are useful." But it seems that it does carry with it an assumption that there are no real categories, that categories change according to the needs, and are only useful in describing things. But the underlying nature of reality is of a continuum.

This seems to be, essentially, that models are useful for prediction, but prediction of unknowns in the data instead of prediction of future system dynamics.

Or how to interpret that data in the light of complex systems.

But how do you tell the value of an explanation? Should it not empower you to some new action or ability? It could be that the explanation is somewhat of a by-product of other prediction-making theories (like how plate tectonics relies on thermodynamics, fluid dynamics, and rock mechanics, which do make predictions).

It might also make predictions itself, such as that volcanoes not on clear plate boundaries might be somehow different (distribution of occurrence over time, correlation with earthquakes, content of magma, size of eruption...), or that understanding the explanation for lightning allows prediction that a grounded metal pole above the house might protect the house from lightning strikes. This might be a different kind of prediction, though, since it isn't predicting future dynamics. Knowing how epidemics works doesn't necessarily allow prediction of total infected counts or length of infection, but it does allow prediction of minimum vaccination rates to avert outbreaks.

Nonetheless, a theory as a tool to explain, with very poor predictive ability, can still be useful, though less valuable than one that also makes testable predictions.

But in general, it seems like data -> theory is the explanation. Theory -> data is the prediction. The strength of the prediction depends on the strength of the theory.

I think that license compatibility is one of the least user friendly areas in the Creative Commons process. Opening resources while being attributed sounds appealing to educators who are dipping their toes in these concepts. Then we pull out Compatibility Charts and people want to run for the hills! I think that the democracy and openness that Creative Commons embodies should be inclusive and I think it's hard for people to decipher these equations which are so crucial to responsible use.

This article provides a well diagrammed and full explanation of the addie model and its' application to technology.

Also included on the site is a link to an online course delivered via diversityedu.com

RATING: 4/5 (rating based upon a score system 1 to 5, 1= lowest 5=highest in terms of content, veracity, easiness of use etc.)

In this pdf, there are many helpful tips and techniques in creating a foundation for technology. The introduction of model strategies are laid out with lots of supporting detail and examples and weblinks. It includes nearly 400 pages of peer-reviewed lessons, models and various strategies.

RATING: 5/5 (rating based upon a score system 1 to 5, 1= lowest 5=highest in terms of content, veracity, easiness of use etc.)

A bit surprised by this “recent”. By that time, much of what has been lumped under the “Web 2.0” umbrella had already shifted a few times. In fact, the “Web 3.0” hype cycle was probably in the “Trough of Disillusionment” if not the Gartner-called “Slope of Enlightenment”.

In context, these other dimensions of “online presence” in Higher Education take a special meaning. Reminds me of WPcampus. One might have thought that it was about using WordPress to enhance learning. While there are some presentations on leveraging WP as a kind of “Learning Management System”, much of it is about Higher Education as a sector for webwork (-development, -design, etc.).

In some ways, the strongest indictment of the MOOC hype.

Interesting list of clients.

This from a university which had dropped Blackboard for iTunes U.

A bit of context.

Sounds a bit like what Clay Shirky was trying to say about the Napster moment coming to Higher Education, five years ago. Skimmed the critique of Shirky’s piece and was mostly nodding in agreement with it. But there might be a discussion about industries having learnt from the Napster moment. After all, the recording industry has been able to withstand this pressure for close to twenty years. Also sounds like this could be a corollary to Chris Anderson’s (in)famous promotion of the “free” (as in profit) model for businesses, almost ten years ago. In other words, we might live another reshaping of “free” in the next 9-10 years.

The presence and impact of cultural aspects which facilitate the resilience to changes in the Nenets’ socio-ecological systems is discussed in further detail in the following piece.

Bruce C. Forbes, “Cultural Resilience of Social-ecological Systems in the Nenets and Yamal- Nenets Autonomous Okrugs, Russia: A Focus on Reindeer Nomads of the Tundra.” Ecology & Society 18, no. 4 (December 2013): 1-16. GreenFILE, EBSCOhost (accessed March 26, 2017).

The inconvenient reality of open source: hungry devs.

My thoughts on Climatic Associations of British Species Distributions Show Good Transferability in Time but Low Predictive Accuracy for Range Change by Rapacciuolo et al. (2012).

It's interesting that the ensembles perform best overall but more poorly for predicting changes in occupancy. It seems possible that ensembling multiple methods is basically resulting in a more static prediction, i.e., something closer to a naive baseline.

This will be an ongoing challenge for species distribution modeling, because most of the data appropriate for these purposes is not collected in such a way as to allow the straightforward application of standard detection probability/occupancy models. This could potentially be addressed by developing models for detection probability based on species and habitat type. These models could be built on smaller/different datasets that include the required data for estimating detectability.

This highlights the importance of comparing forecasts to baseline predictions to determine the skill of the forecast vs. the basic stability of the pattern.

This suggests that it will be difficult to know if a forecast for a particular species will be good or not, unless a model is developed that can predict which species will have what forecast qualities.

This is a really important result and my favorite figure in this ms. For cells that changed occupancy status (e.g., a cell that has occupied at t_1 and was unoccupied at t_2) most models had about a 50% chance of getting the change right (i.e., a coin flip).

Simple unweighted ensembles performed best in this comparison of forecasts from SDMs for 1823 species.

This is arguably the single most important point in this paper. It is equivalent to comparing forecasts to simple baseline forecasts as is typically done in weather forecasting. In weather forecasting it is typical to talk about the "skill" of the forecast, which is how much better it does than a simple baseline. In this case the the baseline is a species range that doesn't move at all. This would be equivalent to a "naive" forecast in traditional time-series analysis since we only have a single previous point in time and the baseline is simply the prediction based on this value not changing.

The point that providing better/worse fits on held out spatial training data is not the same was providing better forecasts is important especially given the argument about simpler models having better transferability.

The lack of dynamic land cover data is a challenge for most SDM and certainly for SDM validation using historical data. If would be interesting to know, in general, how much better modern SDMs become based on held out data when land cover is included.

This restriction to Great Britain for the model building is a meaningful limitation since Great Britain will typically represent a small fraction of the total species range for many of the species involved. However this is a common issue for SDMs and so I think it's a perfectly reasonable choice to make here given the data availability. It would be nice to see this analysis repeated using alternative data sources that cover spatial extents closer to that of the species range. This would help determine how well these results generalize to models built at larger scales.

How so?

I think this is achievable, if we emphasize specialized libraries and cooperative models around resources (i.e. tool/tech libraries, food banks/co-ops)

To me, the most interesting about this program is that it sounds like it’s targeting post-secondary institutions. There are multiple programs to “teach kids to code”. Compulsory education (primary and secondary) can provide a great context for these, in part because the type of learning involved is so broad and pedagogical skills are so recognized. In post-secondary contexts, however, there’s a strong tendency to limit coding to very specific contexts, including Computer Science or individual programs. We probably take for granted that people who need broad coding skills can develop them outside of their college and university programs. In a way, this isn’t that surprising if we’re to compare coding to very basic skills, like typing. Though there are probably many universities and colleges where students can get trained in typing, it’s very separate from the curriculum. It might be “college prep”, but it’s not really a college prerequisite. And there isn’t that much support in post-secondary education. Of course, there are many programs, in any discipline, giving a lot of weight to coding skills. For instance, learners in Digital Humanities probably hone in their ability to code, at some point in their career. And it’s probably hard for most digital arts programs to avoid at least some training in programming languages. It’s just that these “general” programs in coding tend to focus almost exclusively on so-called “K–12 Education”. That this program focuses on diversity is also interesting. Not surprising, as many such initiatives have to do with inequalities, real or perceived. But it might be where something so general can have an impact in Higher Education. It’s also interesting to notice that there isn’t much in terms of branding or otherwise which explicitly connects this initiative with colleges and universities. Pictures on the site show (diverse) adults, presumably registered students at universities and colleges where “education partners” are to be found. But it sounds like the idea of a “school” is purposefully left quite broad or even ambiguous. Of course, these programs might also benefit adult learners who aren’t registered at a formal institution of higher learning. Which would make it closer to “para-educational” programs. In fact, there might something of a lesson for the future of universities and colleges.

To me, the most interesting about this program is that it sounds like it’s targeting post-secondary institutions. There are multiple programs to “teach kids to code”. Compulsory education (primary and secondary) can provide a great context for these, in part because the type of learning involved is so broad and pedagogical skills are so recognized. In post-secondary contexts, however, there’s a strong tendency to limit coding to very specific contexts, including Computer Science or individual programs. We probably take for granted that people who need broad coding skills can develop them outside of their college and university programs. In a way, this isn’t that surprising if we’re to compare coding to very basic skills, like typing. Though there are probably many universities and colleges where students can get trained in typing, it’s very separate from the curriculum. It might be “college prep”, but it’s not really a college prerequisite. And there isn’t that much support in post-secondary education. Of course, there are many programs, in any discipline, giving a lot of weight to coding skills. For instance, learners in Digital Humanities probably hone in their ability to code, at some point in their career. And it’s probably hard for most digital arts programs to avoid at least some training in programming languages. It’s just that these “general” programs in coding tend to focus almost exclusively on so-called “K–12 Education”. That this program focuses on diversity is also interesting. Not surprising, as many such initiatives have to do with inequalities, real or perceived. But it might be where something so general can have an impact in Higher Education. It’s also interesting to notice that there isn’t much in terms of branding or otherwise which explicitly connects this initiative with colleges and universities. Pictures on the site show (diverse) adults, presumably registered students at universities and colleges where “education partners” are to be found. But it sounds like the idea of a “school” is purposefully left quite broad or even ambiguous. Of course, these programs might also benefit adult learners who aren’t registered at a formal institution of higher learning. Which would make it closer to “para-educational” programs. In fact, there might something of a lesson for the future of universities and colleges.

Sign of the times? One part is about economics as the discipline of decision-making. Economists often claim that their work is about any risk/benefit analysis and isn’t purely about money. But the whole thing is still about “resources” or “exchange value”, in one way or another. So, it could be undue influence from this way of thinking. A second part is that, as this piece made clear at the onset, “education is big business”. In some ways, “education” is mostly a term for a sector or market. Schooling, Higher Education, Teaching, and Learning are all related. Corporate training may not belong to the same sector even though many of the aforementioned EdTech players bet big on this. So there’s a logic to focus on the money involved in “education”. Has little to do with learning experiences, but it’s an entrenched system.

Finally, there’s something about efficiency, regardless of effectiveness. It’s somewhat related to economics, but it’s often at a much shallower level. The kind of “your tax dollars at work” thinking which is so common in the United States. “It’s the economy, silly!”

Useful breakdown. Research, predictive models, and recognition are quite distinct from one another and the approaches to data that they imply are quite different. In a way, the “personalized learning” model at the core of the second topic is close to the Big Data attitude (collect all the things and sense will come through eventually) with corresponding ethical problems. Through projects vary greatly, research has a much more solid base in both ethics and epistemology than the kind of Big Data approach used by technocentric outlets. The part about recognition, though, opens the most interesting door. Microcredentials and badges are a part of a broader picture. The data shared in those cases need not be so comprehensive and learners have a lot of agency in the matter. In fact, when then-Ashoka Charles Tsai interviewed Mozilla executive director Mark Surman about badges, the message was quite clear: badges are a way to rethink education as a learner-driven “create your own path” adventure. The contrast between the three models reveals a lot. From the abstract world of research, to the top-down models of Minority Report-style predictive educating, all the way to a form of heutagogy. Lots to chew on.

Sounds fairly damning for a business based on helping diverse students with the labour market…

The Others.

At the risk of sounding cynical (which is a very real thing with annotations), reaching a global market can be very imperialistic a move, regardless of who makes it.

Much hiding in this passing comment. As adjuncts, our contributions to the system are perceived through the exploitation lens.

Though that may not really be the core argument of the piece, it’s more than a passing point. Watters’s raising awareness of this other type of “military-industrial complex” could have a deep impact on many a discussion, including the whole hype about VR (and AR). It’s not just Carnegie-Mellon and Paris’s Polytechnique («l’X») which have strong ties to the military. Or (D)ARPANET. Reminds me of IU’s Dorson getting money for the Folklore Institute during the Cold War by arguing that the Soviets were funding folklore. Even the head of the NEH in 2000 talked about Sputnik and used the language of “beating Europe at culture” when discussing plans for the agency. Not that it means the funding or “innovation” would come directly from the military but it’s all part of the Cold War-era “ideology”. In education, it’s about competing with India or Finland. In other words, the military is part of a much larger plan for “world domination”.

Don’t know what the stats are for “non-profit universities and colleges” but it does feel like an increasing portion of their budgets go to marketing, advertising, PR, and strategic positioning (at least in the United States and Canada).

Somewhat surprising, in the current context.

In education as in health, things get really complex when people have an incentive for people not to improve.

Easier to do with a strict model for success.

Though it’s not exactly the same thing, you could call this “instrumental” or “pragmatic”. Of course, you could have something very practical to amplify learning, and #EdTech is predicated on that idea. But when you do, you make learning so goal-oriented that it shifts its meaning. Very hard to have a “solution” for open-ended learning, though it’s very easy to have tools which can enhance open approaches to learning. Teachers have a tough time and it doesn’t feel so strange to make teachers’ lives easier. Teachers typically don’t make big purchasing decisions but there’s a level of influence from teachers when a “solution” imposes itself. At least, based on the insistence of #BigEdTech on trying to influence teachers (who then pressure administrators to make purchases), one might think that teachers have a say in the matter. If something makes a teaching-related task easier, administrators are likely to perceive the value. Comes down to figures, dollars, expense, expenditures, supplies, HR, budgets… Pedagogy may not even come into play.

Valuing the teaching profession.

Wish they said more. No-cost solutions are neat for one-offs, but pedagogues should be wary of building their practice on services which may start requiring payment.

Time-sensitive, mission-critical, just-in-time, realtime, 24/7…

No.

While this may be true, it sure is sad. Especially as the emphasis on cost is likely to have negative impacts in the long run.

You mean it’s not just about the price of textbooks??

But, but… Eric said it wasn’t marketing!

One sure does.

How many units of learning per dollar?

In direct opposition with the model for most universities, these days. So that may be the fork in the road. But there are more than two paths.

Who came up with such a scheme? A mad scientist? We’re far from Bologna.

There might be something about the connection between learning and “health & wellness”.

And the shift away from “access to content” is still going on, a decade and a half after Napster. If education is a “content industry” and “content industries” are being disrupted, then education will be disrupted… by becoming even more “industrial”.

The battle is raging on, but the issue is predetermined.

One linear model, represented in something of a spiral… Agreed that the transformative experience is tough to “disrupt”, but the whole “content delivery” emphasis shows that the disruption isn’t so quick.

There are a few key cases, here. a) Public Education (much of the planet) b) Parent-Funded Higher Education (US-centric model) c) Corporate Training (emphasis for most learning platforms, these days) d) For-Profit Universities (Apollo Group and such) e) xMOOCs (learning as a startup idea, with freemium models) f) Ad-Supported Apps & Games (Hey! Some of them are “educational”!)

The idea that there’s a “Great Unbundling of (Higher) Education” needs not be restricted to the business side of things, but it’s partly driven by those who perceive education as an “industry”. Producing… graduates?

More consolidation in the LMS market?

In all the platform news, we can talk about “learning management” in view of instructional and course design. But maybe it even goes further than design into a variety of practices which aren´t through-designed.

Important claim, probably coming from the need for reports which answer the “But does it scale?” question.

The reverse of the typical “goal displacement”. Instead of focusing on ensuring our continued employment as “instructors”, we want to make sure learning happens. Deep down, we know we’ll find ways to work, no matter what happens. The comparison with health can be interesting. If doctors had an incentive to keep people sick, society wouldn’t benefit much. Allegedly, Chinese healthcare provides incentives for doctors to help people stay healthy. Sounds like it’d make sense, somehow. Yet education and health are both treated like industries. We produce graduates, future employees, etc. Doctors produce people who fit a pattern of what it means to be healthy in a given social context. There’s even a factory-chain metaphor used when some people apply “lean management” to hospitals or colleges. Not that the problem is with the management philosophy itself. But focusing so much on resource allocation blinds us from a deep reality: as we are getting healthier and more “learned”, roles are shifting.

(This is entirely peripheral to the subject of the article. I am making note of it because I have barely heard of Twitch until recently.)