5 Matching Annotations
  1. Oct 2021
  2. jon-e.net jon-e.net
    Decentralized Infrastructure for (Neuro)science
    5
    1. talmo 16 Oct 2021
      This type of database is natural, expressive, and powerful — for the researchers they are designed for. While some of these databases allow open data submission, they often require explicit moderation and approval to maintain the guaranteed consistency of the database, which can hamper mass use.

      Would a simple workaround be a two-tiered submission system -- like "mod reviewed" / "unreviewed", so you get your URI right after upload, but it won't be search listed until it's verified to pass community standards?

      someone needs to look at stuff at some point! otherwise we have to rely on overly strict schema.

    2. talmo 16 Oct 2021
      The interactions with the other pieces of infrastructure that may induce and incentivize conversion will come later.

      With behavior, this could easily be encouraged by having data repos, model zoos and implementations for a data layer across existing tools.

      In general though, this is a crucial point -- it takes a lot of work to convert to HDF5/NWB. Is the goal just to be able to share it? It doesn't make it more compatible with other tools since there's not enough adoption on that end. It's often even less efficient than a bespoke format -- so who's gaining anything here?

    3. talmo 16 Oct 2021
      a storage implementation in hdf5, and an API for interacting with the data.

      Definitely where things start to get difficult for users. I think that if they made the schema <-> implementation <-> API layer easier, for example, by allowing for other implementations than HDF5, it would make it much more accessible. For example, flat file structures are almost 1:1 mappable onto HDF5, but much easier to produce with less expertise about the intricacies of HDF5.

    4. talmo 16 Oct 2021
      NWB also has an extension interface, which allows, for example, common data sources to be more easily described in the format. These are registered in an extensions catalogue, but at the time of writing it is relatively sparse. The preponderance of lab-specific conversion packages relative to extensions is indicative of an interface and community tools problem: presumably many people are facing similar conversion problems, but because there is not a place to share these techniques in a human-readable way, the effort is duplicated in dispersed codebases. We will return to some possible solutions for knowledge preservation and format extension when we discuss tools for shared knowledge.

      100% -- NWB needs something like conda-forge, where you can make the extension globally accessible through a simple PR. then people can install the extension when they install third party stuff like SLEAP or CaIman which would pull in the extension packages.

    Visit annotations in context

    Annotators

    URL

    jon-e.net/infrastructure