The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure.

most practitioners use a procedure called stochastic gradient descent (SGD).

, The chain rule of derivatives tells us how two small effects (that of a small change of x on y, and that of y on z) are composed.

The backpropagation procedure to compute the gradient of an objective function with respect to the weights of a multilayer stack of modules is nothing more than a practical application of the chain rule for derivatives.

根据评论区 @山丹丹@啸王 的提醒，更正了一些错误（用斜体显示），在此谢谢各位。并根据自己最近的理解，增添了一些东西（用斜体显示）。如果还有错误，欢迎大家指正。第一个问题：为什么引入非线性激励函数？如果不用激励函数（其实相当于激励函数是f(x) = x），在这种情况下你每一层输出都是上层输入的线性函数，很容易验证，无论你神经网络有多少层，输出都是输入的线性组合，与没有隐藏层效果相当，这种情况就是最原始的感知机（Perceptron）了。正因为上面的原因，我们决定引入非线性函数作为激励函数，这样深层神经网络就有意义了（不再是输入的线性组合，可以逼近任意函数）。最早的想法是sigmoid函数或者tanh函数，输出有界，很容易充当下一层输入（以及一些人的生物解释balabala）。第二个问题：为什么引入Relu呢？第一，采用sigmoid等函数，算激活函数时（指数运算），计算量大，反向传播求误差梯度时，求导涉及除法，计算量相对大，而采用Relu激活函数，整个过程的计算量节省很多。第二，对于深层网络，sigmoid函数反向传播时，很容易就会出现梯度消失的情况（在sigmoid接近饱和区时，变换太缓慢，导数趋于0，这种情况会造成信息丢失，参见 @Haofeng Li 答案的第三点），从而无法完成深层网络的训练。第三，Relu会使一部分神经元的输出为0，这样就造成了网络的稀疏性，并且减少了参数的相互依存关系，缓解了过拟合问题的发生（以及一些人的生物解释balabala）。当然现在也有一些对relu的改进，比如prelu，random relu等，在不同的数据集上会有一些训练速度上或者准确率上的改进，具体的大家可以找相关的paper看。多加一句，现在主流的做法，会在做完relu之后，加一步batch normalization，尽可能保证每一层网络的输入具有相同的分布[1]。而最新的paper[2]，他们在加入bypass connection之后，发现改变batch normalization的位置会有更好的效果。大家有兴趣可以看下。

Unsupervised Learning of 3D Structure from Images Authors: Danilo Jimenez Rezende, S. M. Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, Nicolas Heess (Submitted on 3 Jul 2016) Abstract: A key goal of computer vision is to recover the underlying 3D structure from 2D observations of the world. In this paper we learn strong deep generative models of 3D structures, and recover these structures from 3D and 2D images via probabilistic inference. We demonstrate high-quality samples and report log-likelihoods on several datasets, including ShapeNet [2], and establish the first benchmarks in the literature. We also show how these models and their inference networks can be trained end-to-end from 2D images. This demonstrates for the first time the feasibility of learning to infer 3D representations of the world in a purely unsupervised manner.

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning as standard practice for improved new task performance.

encourages students to “steal” and cite ideas from each other’s hypothes.is annotations

Perceptual Learning - Training the brain to better perceive the information it is getting from the eyes.

“'Makerspace' is just a modern term for project-based learning,” he said, adding embracing this teaching style in today’s classroom is “a part of good pedagogy.”

If only 2% – 5% of all faculty and their students (who are doing renewable assignments) were active creators and improvers of OER, that would likely be sufficient.

«Les professeurs qui publient dans une revue disciplinaire n'ont pas toujours le temps, ni la reconnaissance, pour publier dans d'autres publications sur leurs projets ou leurs innovations pédagogiques, explique Anastassis Kozanitis. S'ils le font, ces publications hors discipline ne sont pas reconnues pour leurs demandes de subvention. C'est un frein majeur à la diffusion des recherches dans le domaine au Canada.»

Simply put, we can’t keep preparing students for a world that doesn’t exist

but the better question is whether the form of learning and knowledge-making we are instilling in our children is useful to their future.”

Most of the universities combined the software with human instruction, but a few courses were delivered entirely online.

nothing we did is visible to our analytics systems

In this Discussion blog you will find: #DevtIDEAS Debates videos and summaries (a series of live online ‘webinars’ that brought several practitioners and researchers to debate and share new ideas), editorials from key global international development researchers and and practitioners, and a collection of posts that feature multimedia videos and graphics.

Volunteer Coaches recruit teams of girls to work with female mentors.

It shifted its work to faculty-driven initiatives.

learning agenda on learning analytics

Learning analytics cannot be left to the researchers, IT leadership, the faculty, the provost or any other single sector alone.

An executive at a large provider of digital learning tools pushed back against what he saw as Thille’s “complaint about capitalism.”

a debate over who should control the field of learning analytics

What teachers want in a data dashboard

timely

Data “was something you would use as an autopsy when everything was over,” she said.

Low-shot visual object recognition

While generally misused today, analytics can (theoretically) be used to predict and personalize many facets of teaching & learning, inc. pace, complexity, content, and more.

Connected Course Design at Virginia Commonwealth University

educators and students alike have found themselves more and more flummoxed by a system that values assessment over engagement, learning management over discovery, content over community, outcomes over epiphanies

more student engagement beyond the walls of a school.

I have Serious Rant-y Thoughts on requiring that students inhabit public spaces in professional contexts, and I do wonder how much a class hashtag is useful beyond self-promotion of the course and its amazing instructor.

Atl=xtifl= 0MAXPOOL(RELU(CONV(Etl1)))l >0(1)^Atl=RELU(CONV(Rtl))(2)Etl= [RELU(Atl^Atl);RELU(^AtlAtl)](3)Rtl=CONVLSTM(Et1l;Rt1l;Rtl+1)(4)

Unsupervised convolutional neural networks for motion estimation

the LMS is a platform students will never again use outside of school

similar to picking texts for the course well in advance

“learners must be actively engaged in learning” to achieve deep understanding (Barkley, Cross, & Major, 2005, p. 10).

the algorithm was somewhat more accurate than a coin flip

Both Udacity and Knewton require the human, the learner, to become a technology, to become a component within their well-architected software system. Sit and click. Sit and click.

Identifying issues important in their lives and community, and deciding on one to address

High unemployment■■Racial discrimination■■Neighborhood violence■■Deportation of undocumented immigrants■■High cost of college attendance■■Juvenile justice

To borrow a phrase from libraries and archives, how do we get to a point where we curate connections rather than curating content?

Mistakes are not just opportunities for learning; they are, in an important sense, the only opportunity for learning or making something truly new. Before there can be learning, there must be learners. There are only two non-miraculous ways for learners to come into existence: they must either evolve or be designed and built by learners that evolved. Biological evolution proceeds by a grand, inexorable process of trial and error — and without the errors the trials wouldn’t accomplish anything.

My experimentation with open pedagogy – and my attempts to guide students’ learning with/in and across open platforms – was a social endeavor that invited reciprocal networking.

Generally the literatures related to transformational learning hinge on active student engagement in the learning process and on students assuming responsibility for their learning. Transformative learning, self-directed learning, experiential learning, and collaborative learning, each of which aims to enhance students’ engagement,are some of the pedagogical approaches that are widely described and evaluated in the literature. In addition to active student engagement, another key feature of transformational learning is transformational teaching. In order for students’ role to change, the role and responsibility of faculty must change as well.

Researchers have tracked student emotions while using Crystal Island–a game-based learning environment– and used that research to predict how students will react in other learning situations

The Student-Centered Lecture

A PLN is a self-directed system meant to support lifelong learning through the development, maintenance, and leveraging of digital networks.

Connected Learning is based on the notion that learning is about expanding the connections between people and information within a learner's personal network

Providing Transformative Personalized Student Learning Opportunities as a Path Toward Equity

We are naturally creative and curious. We just have to build systems that nurture our inherent abilities. Schools do not do that.

networked discovery of connections would be at the center of both the learning environment as designed by faculty and the learning environment as experienced by students

the clumsy and often linear arrangements often impede students from digesting each other’s comments and engaging in organic dialogue.

Small Blog On Networked Learning

What is “Connected Learning?”

Koranic school

Do adults (and which adults) have the resources necessary to pursue learning opportunities?

the use of networks is gradually nudging aside more traditional problem-solving approaches based on the marketplace and the choice of a small leadership group or “hierarchies.”

As Fung explains, wicked problems require “multi-sectoral problem-solving” and ways to remove the barriers to “pooling knowledge and coordinating action” through the formation of networks that connect organizations.7

And this leads me to another thought- it seems like in our field there is this desire to go big, to scale, to teach hundreds of thousands, to affect an entire sector. Scale at the dimension is really only achieved by a process of mass duplication where the level of heart-felt connectivity is probably low.

Team-based, contextualized learning underlies the instruction for each track;

“We wanted to create threads between courses so that students can understand connections,”

Ranty Blog Post about Big Data, Learning Analytics, & Higher Ed

A Framework for Identifying and Evaluating Openly Networked Connected Learning Course Designs

Connected Learning & Integrative Thinking: Teaching History at Virginia Tech

True personalized learning calls for a "rethinking and redesign" of schools, which could require them to overhaul classroom structures and schedules, curricula, and the instructional approaches of teachers, Mr. Calkins of EDUCAUSE argued. For instance, in an effective personalized learning model, teachers' roles are more like those of coaches or facilitators than "content providers," he said.

"Technology can help provide students with more choices on how they're going to learn a lesson," Ms. Patrick said. "[It] empowers teachers in personalizing learning" and "empowers students through their own exercise of choice."

Many times, the work we do as educators is actually taking away some of the most powerful learning from our students.

The future of learning analytics, a future in which it is not passé, is one in which learning comes before management

Crafting Connected Courses

Rhizomatic learning as an open pedagogy

Most recently I have been learning from two new-to-me online communities of practice – Wattpad for Writers and DeviantArt for Artists. Their online designs and supportive networked ways of working prompt me to continue thinking about the power of open ways of working in such communities.

gender neutrality, creativity, imagination and tinker time are the basis for learning

Regarding the major obstacles for higher education, blending formal and informal learning is considered one of the solvable challenges

Patrick Ball—a data scientist and the director of research at the Human Rights Data Analysis Group—who has previously given expert testimony before war crimes tribunals, described the NSA's methods as "ridiculously optimistic" and "completely bullshit." A flaw in how the NSA trains SKYNET's machine learning algorithm to analyse cellular metadata, Ball told Ars, makes the results scientifically unsound.

“Search is the cornerstone of Google,” Corrado said. “Machine learning isn’t just a magic syrup that you pour onto a problem and it makes it better. It took a lot of thought and care in order to build something that we really thought was worth doing.”

Pictures of Philosophy

You don’t have to do everything yourself!

Theprincipleismerelythisthatdifferentsubjectsandmodesofstudyshouldbeundertakenbypupilsatfittingtimeswhentheyhavereachedtheproperstageofmentaldevelopment.

There is no single correct way to implement personalized learning

We analyzed the relationship between students’ level of online engagement, and traditional learning metrics, to understand the effectiveness of discussion forums in the context of flipped classrooms.

most e-learning systems

Dweck’s message is that we can’t just adopt a growth mindset and forget about it, and simply praising effort regardless of actual progress is completely counterproductive. Successfully cultivating a growth mindset is an ongoing process that consists of teaching strategies for growth and praising effort thoughtfully, rather than regardlessly.

n and d

Offering students the possibility of experiential learning in personal, interactive, networked computing—in all its gloriously messy varieties—provides the richest opportunity yet for integrative thinking within and beyond "schooling."

Moreover, the experience of building and participating within a digitally mediated network of discovery is itself a form of experiential learning, indeed a kind of metaexperiential learning that vividly and concretely teaches the experience of networks themselves.

Go into your nearest college or university library. Ignore the computer stations and the digital affordances. Enter the stacks, and run your fingers along the spines of the books on the shelves. You're tracing nodes and connections. You're touching networked learning—slow-motion and erratic, to be sure, but solid and present and, truth to tell, thrilling.

“The chief trick to making good mistakes is not to hide them — especially not from yourself.”

Apprendre à programmer permet aux enfants un nouveau rapport aux technologies: de consommateur interactif de manuels scolaires numérisés à la capacité de créer des ressources éducatives numériques et même des mini-jeux.

The whole organic nature of learning experience through the #walkmyworld learning events meant that I learned what I needed to learn as I needed to learn it. It wasn’t a top down dictate of learning outcomes because the outcomes were determined by the process. It is a revolutionary concept — yet as ancient as Aristotle. Learning should never be measured solely by standard outcomes; people learn, and I mean really LEARN, when they discover for themselves what they know, what they want to know, and how they want to know it.

“The key is deliberate practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again. There appear to be no real shortcuts: even Mozart, who was a musical prodigy at age 4, took 13 more years before he began to produce world-class music.”

focus groups where students self-report the effectiveness of the materials are common, particularly among textbook publishers

It’s educators who come up with hypotheses and test them using a large data set.

learned from experience

a good example of the kind of insight that big data is completely blind to

I will investigate the details on this, including the relevant contractual clauses, when I get the chance.

taking a swipe at Knewton

they are making a bet against the software as a replacement for the teacher and against big data

a set of algorithms designed to optimize the commitment of knowledge to long-term memory

numbers have to be interpreted by those who do the daily work of classroom teaching

The challenge, of course, is how to balance concerns of the Hawthorne effect with privacy.

Regular readers know that I am a big fan of the standard-in-development.

If Knewton were doing more of this, I wouldn’t be as critical.

purchasable à la carte

Personalized course materials made possible by real data

Soon, data will make learning outcomes for these courses highly transparent

to offer just lectures

when outcomes are non-transparent

Unbundling has played out in almost every media industry.

wring out every ounce of performance

increased investment in professional development and teaching-friendly tenure and promotion practices

OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.

Big Sur is our newest Open Rack-compatible hardware designed for AI computing at a large scale. In collaboration with partners, we've built Big Sur to incorporate eight high-performance GPUs

The most popular project for the MUA to tackle was Learning Analytics

Lambda Solutions [Corrected.]

help build a community that functions as a collective pool of knowledge and possibility

literary Facebook