A resource can map to the empty set, which allowsreferences to be made to a concept before any realization ofthat concept exist
This is a very useful but underutilized property. It allows you to e.g. announce in advance that a resource will exist at some point in the future, and thereby effectively receive "updates" to the linking document without requiring changes to the document itself.
Over the past few years, many “efficient Trans-former” approaches have been proposed that re-duce the cost of the attention mechanism over longinputs (Child et al., 2019; Ainslie et al., 2020; Belt-agy et al., 2020; Zaheer et al., 2020; Wang et al.,2020; Tay et al., 2021; Guo et al., 2022). However,especially for larger models, the feedforward andprojection layers actually make up the majority ofthe computational burden and can render process-ing long inputs intractable
Recent improvements in transformers for long documents have focused on efficiencies in the attention mechanism but the feed-forward and projection layers are still expensive for long docs