- Mar 2019
-
csus-dspace.calstate.edu csus-dspace.calstate.edu
-
DEEP LEARNING WITH CONVOLUTIONAL NEURAL NETWORKS FOR IMAGE RECOGNITION: STEP-BY-STEP PROCESS FROM PREPARATION TO GENERALIZATION
-
-
scholarworks.sjsu.edu scholarworks.sjsu.edu
-
Deep Learning for Chatbots
Tags
Annotators
URL
-
-
stacks.stanford.edu stacks.stanford.edu
-
EFFICIENT METHODS AND HARDWARE FOR DEEP LEARNING
Deep Compression" can reduce the model sizeby 18?to 49?without hurting the prediction accuracy. We also discovered that pruning and thesparsity constraint not only applies to model compression but also applies to regularization, andwe proposed dense-sparse-dense training (DSD), which can improve the prediction accuracy for awide range of deep learning models. To efficiently implement "Deep Compression" in hardware,we developed EIE, the "Efficient Inference Engine", a domain-specific hardware accelerator thatperforms inference directly on the compressed model which significantly saves memory bandwidth.Taking advantage of the compressed model, and being able to deal with the irregular computationpattern efficiently, EIE improves the speed by 13?and energy efficiency by 3,400?over GPU
-
-
cjc.ict.ac.cn cjc.ict.ac.cn
-
深度文本匹配综述
-
-
github.com github.com
-
all kinds of text classification models and more with deep learning
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification
Tags
Annotators
URL
-
-
www.ijcai.org www.ijcai.org
-
Differentiated Attentive Representation Learning for Sentence Classification
-
-
arxiv.org arxiv.org
-
A Gentle Tutorial of Recurrent Neural Network with ErrorBackpropagation
A Gentle Tutorial of Recurrent Neural Network with ErrorBackpropagation
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
A Tutorial on Deep Latent Variable Models of Natural Language
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
BACKGROUND: ATTENTIONNETWORKS
Tags
Annotators
URL
-
-
-
To the best of our knowl-edge, there has not been any other work exploringthe use of attention-based architectures for NMT
目前并没人来用attention来做机器翻译
Tags
Annotators
URL
-
-
github.com github.com
Tags
Annotators
URL
-
-
-
LSTM Derivations
-
-
jalammar.github.io jalammar.github.io
-
博客很赞！
-
-
arxiv.org arxiv.org
-
One of the challenges of deep learning is that the gradients with respect to the weights in one layerare highly dependent on the outputs of the neurons in the previous layer especially if these outputschange in a highly correlated way. Batch normalization [Ioffe and Szegedy, 2015] was proposedto reduce such undesirable “covariate shift”. The method normalizes the summed inputs to eachhidden unit over the training cases. Specifically, for theithsummed input in thelthlayer, the batchnormalization method rescales the summed inputs according to their variances under the distributionof the data
batch normalization的出现是为了解决神经元的输入和当前计算值交互的高度依赖的问题。因为要计算期望值，所以需要拿到所有样本然后进行计算，显然不太现实。因此将取样范围和训练时的mini-batch保持一致。但是这就把局限转移到mini-batch的大小上了，很难应用到RNN。因此需要LayerNormalization.
-
Layer Normalization
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE
Tags
Annotators
URL
-
-
emnlp2014.org emnlp2014.org
-
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
A STRUCTURED SELF-ATTENTIVE SENTENCE EMBEDDING
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
Contextual Word Representations: A Contextual Introduction
Tags
Annotators
URL
-
-
www.csie.ntu.edu.tw www.csie.ntu.edu.tw寬螢幕簡報1
-
Deep Learning for Dialogue Systems
-
- Feb 2019
-
-
ecent advances of deep learning have inspiredmany applications of neural models to dialoguesystems. Wen et al. (2017) and Bordes et al.(2017) introduced a network-based end-to-endtrainable task-oriented dialogue system, whichtreated dialogue system learning as the problemof learning a mapping from dialogue histories tosystem responses, and applied an encoder-decodermodel to train the whole system
Wen和Bordes介绍了一种基于网络的端到端的任务型对话系统，这个系统将对话系统学习看成是从历史回话到系统回复的映射关系的学习问题，并且应用了一个编码解码器来训练整个系统。
这个思路很有意思，和我之前想构建一个电销员的语料库来做用户回复响应很像。这个很有可能。
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
How to Generate a Good Word Embedding?
Tags
Annotators
URL
-
-
licstar.net licstar.net
-
-
gitee.com gitee.com
-
yann.lecun.com yann.lecun.com
Tags
Annotators
URL
-
-
gitee.com gitee.com
-
book.haihome.top book.haihome.top
-
gitee.com gitee.com
-
dougalmaclaurin.com dougalmaclaurin.com
Tags
Annotators
URL
-
-
gitee.com gitee.com
-
arxiv.org arxiv.org
-
Recent Trends in Deep Learning BasedNatural Language Processing
Tags
Annotators
URL
-
-
gitee.com gitee.com
-
nlp.stanford.edu nlp.stanford.edu
-
GloVe: Global Vectors for Word Representation
Tags
Annotators
URL
-
-
www.jmlr.org www.jmlr.org
-
Natural Language Processing (Almost) from Scratch
-
-
-
Short Text Similarity with Word Embeddings
-
-
gitee.com gitee.com
-
gitee.com gitee.com
-
arxiv.org arxiv.org
-
Attention Is All You Need
Tags
Annotators
URL
-
-
cs.stanford.edu cs.stanford.edu
-
Distributed Representations of Sentences and Documents - Doc2Vec
Tags
Annotators
URL
-
-
tanthiamhuat.files.wordpress.com tanthiamhuat.files.wordpress.com
-
aclanthology.org aclanthology.org
-
Deep contextualized word representations
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
BERT: Pre-training of Deep Bidirectional Transformers forLanguage Understanding
-
- Jan 2019
-
www.sciencedirect.com www.sciencedirect.com
-
By utilizing the Deeplearning4j library1 for model representation, learning and prediction, KNIME builds upon a well performing open source solution with a thriving community.
-
It is especially thanks to the work of Yann LeCun and Yoshua Bengio (LeCun et al., 2015) that the application of deep neural networks has boomed in recent years. The technique, which utilizes neural networks with many layers and enhanced backpropagation algorithms for learning, was made possible through both new research and the ever increasing performance of computer chips.
-
- Dec 2018
-
winstonhsu.info winstonhsu.info
Tags
Annotators
URL
-
-
berkeley-deep-learning.github.io berkeley-deep-learning.github.io
-
bcourses.berkeley.edu bcourses.berkeley.edu
Tags
Annotators
URL
-
-
github.com github.com
Tags
Annotators
URL
-
-
shop.oreilly.com shop.oreilly.com
Tags
Annotators
URL
-
-
github.com github.com
Tags
Annotators
URL
-
-
github.com github.com
-
github.com github.com
Tags
Annotators
URL
-
-
github.com github.com
Tags
Annotators
URL
-
- Oct 2018
-
-
Many detection methods such as Faster-RCNN and YOLO, perform badly in small objects detec-tion. With some considerable improvements in the originalframework of YOLOv2, our proposed SO-YOLO can solvethis problem perfectly.
-
-
ieeexplore.ieee.org ieeexplore.ieee.org
-
As a convolutional neural network, SO-YOLO outperforms state-of-the-art detection methods both in accuracy and speed.
-
SO-YOLO performs well in detecting small objects compared with other methods.
Tags
Annotators
URL
-
- Jul 2018
-
-
President Donald Trump has multiple reasons as to why he should take control of the Federal Reserve. He will do so both because he can and because his broader policies argue that he should do so. The president is anti-overregulating American industry. The Fed is a leader in pushing stringent regulation on the nation. By raising interest rates and stopping the growth in the money supply it stands in the way of further growth in the American economy.
-
-
www.cnbc.com www.cnbc.com
-
Trump has a right to call out the Fed if it's not acting in the US best interest
-
-
www.youtube.com www.youtube.com
-
The Truth About Migrants (2018)
7:37
when this migrant in France scaled a building to save a baby I guess that means we should just get rid of borders and laws why this dude didn't grab the baby probably best not to ask questions
The thumbnail of this video is ominous/portentous . it's a bad omen for women, their dautghters will pay for their mothers' stupidity.
-
- May 2018
-
web.stanford.edu web.stanford.edu
-
CS 20: Tensorflow for Deep Learning Research
课程时间: 1月-3月, 2018
Tags
Annotators
URL
-
- Mar 2018
-
arxiv.org arxiv.org
-
Mathematics of Deep Learning
-
-
distill.pub distill.pub
-
distill.pub distill.pub
Tags
Annotators
URL
-
-
webfoundation.org webfoundation.org
-
artificial neural network
El deep learning incluye redes neuronales
-
Artificial intelligence (AI), machine learning and deep learning
Explicación gráfica de artificial intelligence, machine learning y deep learning
-
- Oct 2017
-
www.lifeextension.com www.lifeextension.com
-
Beta blockers have long been associated with sleep disturbances such as difficulty falling asleep, staying asleep, and insomnia. They have been shown to reduce the production of melatonin via specific inhibition of beta-1 adrenergic receptors. Melatonin is a hormone secreted by the pineal gland in the brain, and helps in maintaining normal circadian rhythms.6,20-21 People with hypertension already have a lower melatonin production rate than those with normal blood pressure.22
The question becomes, then, do beta blockers impair sleep when exogenous melatonin is administered concurrently?
-
- Sep 2017
-
tenso.rs tenso.rs
Tags
Annotators
URL
-
- Aug 2017
-
arxiv.org arxiv.org
-
This is a very easy paper to follow, but it looks like their methodology is a simple way to improve performance on limited data. I'm curious how well this is reproduced elsewhere.
-
-
blog.athelas.com blog.athelas.com
-
Excellent overview. I found the papers a little hard to grasp, and this cleared a lot of that up.
-
-
video.nationalgeographic.com video.nationalgeographic.com
-
Deep in the Pacific Ocean, scientists may have discovered a ghostly new species of snailfish. Snailfish are the deepest dwelling vertebrates on Earth. Some live over 5 miles below the surface. This one was observed at a depth of 1.5 miles, in the Pacific Remote Islands Marine National Monument. It’s possible this is the first time it’s ever been observed by humans.
Cool!
-
- May 2017
-
-
bookmarked for deep learners
Tags
Annotators
URL
-
- Apr 2017
-
www.fast.ai www.fast.ai
-
areas where deep learning is currently being poorly utilized
who is curating a list of deep learning success stories, case studies and applications?
-
highly automated tools for training deep learning models
such as?
-
The best way we can help these people is by giving them the tools and knowledge to solve their own problems, using their own expertise and experience.
Agree or disagree?
Tags
Annotators
URL
-
-
channel9.msdn.com channel9.msdn.com
-
first 15 minutes is a very interesting history of deep learning
Tags
Annotators
URL
-
-
mtldata.com mtldata.com
-
Slack
register or login here: http://mtldata.com/slack
-
r/deeplearners subreddit
-
online reading group
Please share any feedback you have.
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
Appendix A:Table of various deep learning applications
This is a good list. Has anyone come across a comprehensive list of deep learning applications?
-
-
colah.github.io colah.github.io
-
Almost all exciting results based on recurrent neural networks are achieved with them.
lstm是rnn的一种，但是一般所RNN指传统标准的RNN
-
-
www.tensorflow.org www.tensorflow.org
-
If we write that out as equations, we get:
It would be easier to understand what are x and y and W here if the actual numbers were used, like 784, 10, 55000, etc. In this simple example there are 3 x and 3 y, which is misleading. In reality there are 784 x elements (for each pixel) and 55,000 such x arrays and only 10 y elements (for each digit) and then 55,000 of them.
-
- Mar 2017
-
engineering.skymind.io engineering.skymind.io
-
Consequently, our advice is simple: continue to train your networks on a single machine, until the training time becomes prohibitive.
一定要对 数据加载时间、参数通信时间、计算时间有个明确的评估，不能为了并行而并行。能单机解决的问题就不着急上多机。
-
odel parallelism can work well in practice, data parallelism is arguably the preferred approach for distributed systems and has been the focus of more research
why ?
-
- Dec 2016
-
www.youtube.com www.youtube.com
-
Key points:
- Scale of data is especially good for large NNs
- Having a combination of HPC and AI skills is important to have optimal impact (handle scale challenges and bigger/complex NN)
- Most of the value right now comes from CNNS, FCs, RNNS. Unsupervised, GANs and others might be future but they are research topics right now.
- E2E DL might be relevant for some cases in future like speech -> transcript, Image -> captioning, text -> image
- Self driving cars might also move to E2E, but none of us have enough data image -> steer
Workflow:
- Bias = Training error - Human error. Try Bigger model, run longer, New model architecture
- Variance = Dev error - Train error. Try More data, Regularization, New model architecture.
- Conflict between bias and variance is weaker in DL. We can have bigger model with more data.
More data:
- Data synthesis/augmentation is becoming useful and popular: OCR (superpose alphabets on various images), Speech (Superpose various background noises), NLP(?) But does have drawbacks, if it is not representative
- Unified data warehouse helps leverage data usage across company
Data set breakdown:
- Dev and test should come from same distribution. As we spend a lot of time optimizing for Dev accuracy.
Progress plateaus above Human level performance:
- But there is theoretical optimal error rate (Bayes rate)
What to do when bias is high:
- Look at examples of the ones machine got it wrong
- Get labels from humans?
- Error analysis: Segment training - identify segments where training error is higher than human.
- Estimate bias/variance effect?
How do you define human level performance: Example: Error of a panel of experts
Size of data:
- How do you define a NN as small vs medium vs large?
- Is the reason large NN can leverage bigger data is because it would not cause overfitting unlike on smaller NNs?
-
-
karpathy.medium.com karpathy.medium.com
-
A good blog post on why do you need to understand backpropagation links to videos from CS231n by karpathy.
-
- Nov 2016
-
classroom.udacity.com classroom.udacity.com
-
Deep neural networks use multiple layers with each layer requiring it's own weight and bias.
Every layer needs its own weights and bias. And in tensorflow, it is a good practice to put all weights inside a dictionary, which is easier for management.
-
- Oct 2016
-
medium.com medium.com
-
www.miaoerduo.com www.miaoerduo.com
-
这里要求，输入的数据时成对存在，每一对都有一个公共的label，是否是同一个类别。
Verification signal
-
- Jul 2016
-
-
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
xnor-net: a very efficient network
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
Tags
Annotators
URL
-
-
inst-fs-iad-prod.inscloudgate.net inst-fs-iad-prod.inscloudgate.net
-
half-spaces sepa-rated by a hyperplane19.
传统算法的局限，在图像和语音领域，需要对不相干的钝感和对几个很小地方差异的敏感
-
Deep learning
四大金刚中的三个
-
The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure.
深度学习的最重要的一方面就是多层特征自动学习
-
most practitioners use a procedure called stochastic gradient descent (SGD).
随机梯度下降算法，讲的很好
-
, The chain rule of derivatives tells us how two small effects (that of a small change of x on y, and that of y on z) are composed.
我擦！原来如此！！！
-
The backpropagation procedure to compute the gradient of an objective function with respect to the weights of a multilayer stack of modules is nothing more than a practical application of the chain rule for derivatives.
反向传播过程来计算一个具有多层模块权重的目标函数的梯度其实不过是求导链式规则实际应用。
-
-
en.wikipedia.org en.wikipedia.org
-
Run-length encoding (RLE)
数据无损压缩编码
Tags
Annotators
URL
-
-
www.zhihu.com www.zhihu.com
-
根据评论区 @山丹丹@啸王 的提醒，更正了一些错误（用斜体显示），在此谢谢各位。并根据自己最近的理解，增添了一些东西（用斜体显示）。如果还有错误，欢迎大家指正。第一个问题：为什么引入非线性激励函数？如果不用激励函数（其实相当于激励函数是f(x) = x），在这种情况下你每一层输出都是上层输入的线性函数，很容易验证，无论你神经网络有多少层，输出都是输入的线性组合，与没有隐藏层效果相当，这种情况就是最原始的感知机（Perceptron）了。正因为上面的原因，我们决定引入非线性函数作为激励函数，这样深层神经网络就有意义了（不再是输入的线性组合，可以逼近任意函数）。最早的想法是sigmoid函数或者tanh函数，输出有界，很容易充当下一层输入（以及一些人的生物解释balabala）。第二个问题：为什么引入Relu呢？第一，采用sigmoid等函数，算激活函数时（指数运算），计算量大，反向传播求误差梯度时，求导涉及除法，计算量相对大，而采用Relu激活函数，整个过程的计算量节省很多。第二，对于深层网络，sigmoid函数反向传播时，很容易就会出现梯度消失的情况（在sigmoid接近饱和区时，变换太缓慢，导数趋于0，这种情况会造成信息丢失，参见 @Haofeng Li 答案的第三点），从而无法完成深层网络的训练。第三，Relu会使一部分神经元的输出为0，这样就造成了网络的稀疏性，并且减少了参数的相互依存关系，缓解了过拟合问题的发生（以及一些人的生物解释balabala）。当然现在也有一些对relu的改进，比如prelu，random relu等，在不同的数据集上会有一些训练速度上或者准确率上的改进，具体的大家可以找相关的paper看。多加一句，现在主流的做法，会在做完relu之后，加一步batch normalization，尽可能保证每一层网络的输入具有相同的分布[1]。而最新的paper[2]，他们在加入bypass connection之后，发现改变batch normalization的位置会有更好的效果。大家有兴趣可以看下。
ReLU的好
Tags
Annotators
URL
-
-
-
Unsupervised Learning of 3D Structure from Images Authors: Danilo Jimenez Rezende, S. M. Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, Nicolas Heess (Submitted on 3 Jul 2016) Abstract: A key goal of computer vision is to recover the underlying 3D structure from 2D observations of the world. In this paper we learn strong deep generative models of 3D structures, and recover these structures from 3D and 2D images via probabilistic inference. We demonstrate high-quality samples and report log-likelihoods on several datasets, including ShapeNet [2], and establish the first benchmarks in the literature. We also show how these models and their inference networks can be trained end-to-end from 2D images. This demonstrates for the first time the feasibility of learning to infer 3D representations of the world in a purely unsupervised manner.
The 3D representation of a 2D image is ambiguous and multi-modal. We achieve such reasoning by learning a generative model of 3D structures, and recover this structure from 2D images via probabilistic inference.
Tags
Annotators
URL
-
-
arxiv.org arxiv.org
-
When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning as standard practice for improved new task performance.
Learning w/o Forgetting: distilled transfer learning
Tags
Annotators
URL
-
- Jun 2016
-
arxiv.org arxiv.org
-
Low-shot visual object recognition
Tags
Annotators
URL
-
- Apr 2016
-
techcrunch.com techcrunch.com
-
We should have control of the algorithms and data that guide our experiences online, and increasingly offline. Under our guidance, they can be powerful personal assistants.
Big business has been very militant about protecting their "intellectual property". Yet they regard every detail of our personal lives as theirs to collect and sell at whim. What a bunch of little darlings they are.
-
- Dec 2015
-
openai.com openai.comOpenAI1
-
OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.
They're hiring: https://openai.com/about/
-
-
-
let you share a URL that navigates directly to the selection you shared.
An important form of deep linking.
-
- Jul 2015
-
www.csdl.tamu.edu www.csdl.tamu.edudl97.pdf1
-
Levy, D.M. (1997) "I read the news today oh boy".
Check out this article and share with JD.
Tags
Annotators
URL
-
- Jun 2015
-
www.meneame.net www.meneame.net
-
¿Podría contar alguna anécdota de su participación en el desarrollo de Deep Blue? Hay muchas anécdotas, pero son largas de contar. Solo te diré que el día que empecé a trabajar para IBM para enumerar los puntos débiles de Kaspárov el folio se quedó en blanco.
-