So, basically, compact means bounded and closed?

But what if the cost function (reward function) is not available? Or it is usually available for many cases of interest?

An protocol between an object and user's input.

be aware of confirmation bias

Start small in writing your aims!

Test the basic assumption will be of great value to the field.

When threads inside a warp branches to different execution paths. Instead of all 32 threads in the warp do the same instruction, on average only half of the threads do the instruction when warp divergence occurs. This causes 50% performance loss.

an important take-home message for dealing with branch divergence.

This statement conflicts the statement that only one warp is executed at a time per SM.

Q: Only one instruction unit is a SM, and a SM has many warps. Does this imply all these warps within the same SM execute the same set of instructions?

A: No. Each SM has a (cost-free) warp scheduler which prioritizes ready warps (along the time dimension). Take a look at the figure in page 6. of http://www.math.ncku.edu.tw/~mhchen/HPC/CUDA/GPGPU_Lecture5.pdf

• world_rank, partner_rank: private for each process?
• ping_pong_count: shared?

-> NO! Each process spawns an instance of the same program with its own memory space. Operations that a process carries out on the variables in its memory space does not affect the values of the variables of another process' memory space. To communicate a value of a variable from one process to another (e.g., update the newly computed value of a variable X to its value in another process), we can use MPI_Send and MPI_Recv.

**After MPI_Send, the sent data is packed into a buffer and the program continues (e.g., it needs no receiver for the program to continue).

However, after MPI_Recv, the program waits until it receives the data. **

cudaMalloc API: first (returning) arg as the address of the pointer to the allocated device memory

L(\theta, \phi; x) is the lower bound of log-likelihood p_{\theta}(x)

CxHXW = 1x1x1?

This is my first note using 'Hypothesis.io'