We already did, it was called Lisp Machines and Smalltalk, and we are yet to fully replicate them.

With computer interfaces, you hardly ever interact with something in an immediate way. I want a comment to appear on this site but instead I am writing this text in a white box and not where the comment would appear. All the computer interactions are mediated by these in-between steps. (An example for unmediated interaction would be cooking. What you chop is what you get

To me, the the most interesting part of Bret Victors ideas lie in the re-embodiment of disembodied interactions. The computers choreograph us anyhow, but often in a very poor and limited way – only our fingertips and our eyes move a bit. Why not make the choreography of interaction richer? Why not create a computer-aided full-body choreography of interaction?