所谓总变差就是两个向量作差然后求二范数

回顾矩阵的秩的定义：设在矩阵\(\mathbf{A}\)中有一个不等于0的\(r\)阶子式\(D\)，且所有\(r+1\)阶子式（如果存在的话）全等于0，那么\(D\)称为矩阵\(\mathbf{A}\)的最高阶非零子式，数\(r\)称为矩阵\(\mathbf{A}\)的秩。

$$x[n] = \sum\limits_{k = -\infty}^\infty \left( \, y_{high}[k] \cdot g[-n + 2k] \, \right) + \left( \, y_{low}[k] \cdot h[-n + 2k] \, \right)$$

Note that the summing variable is now \(k\)

With the highest frequency resolution, lowest time resolution.

For 2D-DWT of an image, the last level of decomposition will look "small" in size (because of the low spatial resolution).

Is it because the range of the frequency resolution has been "shrinked" that we say the frequency resolution is increased?

rpn = F.relu(self.rpn_net(net_conv))

called net_conv in the code

Each neuron has two weight matrix, one for multiplying \(x^{(t)}\) (denoted as \(U\)) and one for multiplying by \(h^{(t-1)}\) (denoted as \(W\)). 4 pairs of \(U\) and \(W\) in total.

or cell unit

that is the \(\Phi\) in \(\mathbf{y}=\Phi \mathbf{x}\) (maybe \(\Phi\) is called measurement matrix elsewhere?)

https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/

Nope, not exactly useless. I believe this is the very basics that lie under the implementation of torch.no_grad()

code in exit

code in enter

wrapper乍一看在do_twice中定义的时候似乎没有传入参数的必要，但仔细一想便会发现wrapper是主程序中事实上被执行的那个（外层）函数，因此As a rule of thumb, wrapper的参数列表应该和被修饰函数一致。于是最普适的方法就是设置参数列表为*args, **kwargs。

实效上在运行时利用decorator提供的“环境”，利用say_whee这个“零件”组装了一个新的函数

This is known as "f-strings" introduced in python 3.6

标准正态分布

I think

$$v_{t+1}=\rho v_t-\alpha\nabla f(x)$$

$$x_{t+1}=x_t+v_{t+1}$$

is received better

实例化某个损失函数，并起了一个泛化的变量名（这里是criterion，你想叫它loss_fn也可以）

把这些score看成未归一化的对数概率

$$v(t)=v_{0}+at$$

$$v(t)=\dot{x}(t)$$

$$x(t)=v_{0}t + \frac{1}{2}at^{2}$$

$$f_t=ma$$

$$x(t)=x(t-1)+v(t-1)\Delta t+\frac{1}{2}a(\Delta t)^2$$

$$v(t-1)=\dot{x}(t-1)$$

a batch is some samples from the whole set of trainning data

i.e. the five neurons in the figure above.

What is the design intuitive to pass a function object here?

Typo. Should be tf.estimator.DNNClassifier

This is also known as a ramp function (斜度函数或者铰链函数) and is analogous to half-wave rectification in electrical engineering. (回忆正弦交流电加在二极管上)

Could you please cite the paper directly, please? I just got a 403 forbidden following this link.

Correct me if I am wrong: there is a typo at line 50, column 58 of file assignment1/cs231n/classifiers/linear_classifier.py. The X_batch matrix should have shape (batch_size, dim) rather than being transposed.

(I am still confused why you make everything transposed of what you have mentioned in the lecture video)

I think it would be clearer if you figure out the expression of gradient, which is written as weights_grad, grad or dW in the assignment code:

dW= $$ \nabla{\mathbf{W}} \mathbf{L} = \frac{\partial L}{\partial \mathbf{W}} = \frac{1}{N} \left( \begin{array}{ccc} \sum{i=1}^{N} \frac{\partial L_i}{\partial \mathbf{w}1} \ \sum{i=1}^{N} \frac{\partial L_i}{\partial \mathbf{w}2} \ \vdots \ \sum{i=1}^{N} \frac{\partial L_i}{\partial \mathbf{w}_c} \end{array} \right)

• \lambda \frac{\partial R(\mathbf{W})}{\partial \mathbf{W}} $$

where N is the number of training samples, and c is the number of classes.

OpenCV Python binding provides an imread as well. Don't mess them up.