Would you argue that e.g. MLP is also a deep net? I would slightly rephrase the term "deep net" and change it into something like "neural network-based classifiers".
But, related to this, I actually found quite a relevant and interesting additional paper! It happened to be published 3 days ago, and the paper basically argued that you don't neccesarily need deep convnets to solve similar societal problems - any problem that can be solved using a transformer can be solved by a MLP/CNN, and vice versa, provided that you do exhaustive tuning and use the right inductive bias. Idem ditto for RNNs!
Link to CNN paper: https://arxiv.org/abs/2201.03545
Link to RNN paper: https://arxiv.org/abs/1611.09913