We know through painful experience that freedom is never voluntarily given by the oppressor; it must be demanded by the oppressed.
King is saying that those who are hurt the most should demand change, not wait for it to happen, since those who are doing the oppressing like the way things are and they won’t change anything.