Shipyard is open to discussing devgrants
shipyard
devgrants
Shipyard is open to discussing devgrants
shipyard
devgrants
Desktop users and home servers can now provide at scale.
Desktop users and home servers -
can now provide at scale. =
Most users should rely on Provide Sweep alone, which delivers efficient content routing without the resource overhead.
converging supporting personal firs
interpersonal networks
Faster lookups
useful for applications requiring faster peer and content discovery
Stale routing:
Provider records may be sent to peers that have gone offline or are no longer closest to the CID, since the snapshot is only refreshed hourly
we recommend disabling
Accelerated DHT Client
Reprovide Sweep vs Accelerated DHT Client metrics

providing millions of CIDs
or more, you should increase the number of provide workers - to handle the higher throughput.
Reprovide Sweep is now the default provider in Kubo v0.39.
Default
the Accelerated client trades continuous crawling overhead for zero-latency lookups
tradeoff
continuous crawling vs zero-latency lookup
The Accelerated DHT Client serves these queries from its cached network map without additional requests.
serves from map lookup no network request
Accelerated is spiky, Reprovide Sweep is regular.
spiky vs regular
an order of magnitude more than Reprovide Sweep.
Reprovide Sweep 132k requests 22 h
vs
Accelerated DHT client 1,87 million requests 22h
Reprovide Sweep successfully reprovides all 100,000 CIDs within the 3-hour window.
100k CIDs within a 3-hour window single thread
While effective at completing the work
this burst approach creates temporary peaks in resource usageβhigh CPU load, network bandwidth consumption, and numerous concurrent connections

Performance Comparison

When the system is ready to perform a provide operation,
it pops all CIDs from the same region as the head of the queue and provides them together in a single batch
Provide Sweep caps the number of lookups at around 3,000, regardless of how many CIDs youβre providing.
Wow caps at 3k lookups
independently of the number of CIDs being provided
since every content is contextualized =
can just lookup adjacent peers by shift of focus
Each region is defined by a keyspace prefix and is sized to contain at least 20 peers, ensuring there are always enough nodes to satisfy the DHTβs replication factor
keyspace prefix
20 peers
replication factor
ransforms what was once a resource-intensive burst
into a smooth, continuous background process that distributes the load over time.
Reprovide Sweep schedules region reprovides evenly across the 22-hour
Reprovide Sweep
organizes all CIDs by keyspace regions and sweeps through these regions sequentially
Metaphysics of Adjacency
Provide Sweep finds them once and delivers all relevant provider records together
will send dozens or even hundreds of provider records to the same servers
lots of records to the same server
pigeonhole principle
if you have far more CIDs to provide than there are DHT servers in the network, then each server must be allocated multiple CIDs. Since each CID must be stored on 20 servers (the replication factor), and there are approximately 10,000 active DHT servers, a node providing hundreds of thousands of CIDs will send dozens or even hundreds of provider records to the same servers.
CIDs that are close together in the Kademlia keyspace share the same closest peers.
CIDs that are close together
mpractical for smaller scale IPFS users who want to participate in the network without such overhead.
inpractical for smaller scale IPFS users
crawling the network is expensive
expensive
consider timebound focus
peer ids of near contacts two hops!
network remains robust even as peers constantly churn in and out
network robust churn
With this snapshot in memory, finding the 20 closest peers to any key becomes instant: no messages, no timeouts, just a local lookup.
lookup
maintain a complete map of the network
complete map of the network
Interpersonal connectivity may help to scale this
scaling salience
content gradually disappears from the DHT
conrifmed
Kubo can only reprovide 5,280 CIDs
kubo can only reprovide 5k CIDs in 22 hours
if new content arrives faster than Kubo can provide it, the queue grows indefinitely.
new content arrive faster
can't keep up
finding 20 reachable closest peers requires patience and retries.
pateience and retries
bottleneck isnβt the actual storage of provider records, it is the lookup process itself.
bottleneck lookup
regular refresh serves two purposes
In my experience it does not seem to work!
Need to pay attention to provide.strategy
matches your provide strategy
focus on Kubo because it supports self-hosting
slef-hosting
is magic
the provide and reprovide mechanisms
indeed key
no dependency on continued funding
Yes
Vulnerable to defunding
defunding
periodic reprovides
key
reproviding would happen oranically with interpersonal traffic
Scales organically with content and participants
scales organcally
provider list unless renewed.
Yes, IPNS records need to be renewed every 48 hours!
ime-to-live (TTL) of 48 hours
TTL 48 hours
the DHT implements an expiration mechanism
a directory linking what youβre looking for to where you can find it.
directory
lightweight pointers that map each CID to a list of peer IDs
smart resume capabilities to handle node restarts gracefully
replication factor of 20
replication factor of 20
The solution is redundancy.
obtainable by interpersonal cross referencing linking and sharing
Key Idea Individuals form and maintain their own autonomous shards!
if nodes storing information about content suddenly go offline, that content becomes undiscoverable.
undiscoerable
ogβ(N) steps
steps
logβ(N) other peers,
peers
Those peers will either return the object itself or point to even closer peers who might have it
closer peer
a mathematical way to calculate distance in this virtual space.
calculate distance
objectβs address in the DHT
link to that private Lcuel Unique Ids that form a Personal Indranet Worspace nidirectionally meaningfully linked strucured deeply meaningfully intentionlly intertwingled spaces enScapes that encapsulate curated associative complexes
reusable repurposable with full provi=enace and history of its own co-evolution and
Conversations that are continuous without being synchronous
custom routing mechanisms
People cantered interest based emergent networks
where connecting People Ideas and Reelevant resources
simultaneously scaling mutual learning, synthesis, and Reach. Searches including the ability to exchange searches and results serendioity engine
Pensieve
a distributed key-value store.
distributed key value store
Private Content Routing
Interpersonal
InterPlanetary Network Indexer
network indexer
providing content at scale practical.
content at scale
distributed system that maps CIDs to the peers that can serve them
map CIDs to peers
discovery problem
Discovery probem
can be further aleviated by weaving meaningfullyintentionally transparently named connections,trails that are marked.
decouples content identity from where it lives on the network
decopulies identity from where it ives
Provide Sweep changes everything
batching CIDs - that get allocated to the same DHT servers and - sweeping through the keyspace systematically,
weβve unlocked dramatic efficiency gains.
For a node providing 100k CIDs, - this means a 97% reduction in lookup operations.
Resource-constrained nodes can now handle - hundreds of thousands of CIDs,
while high-capacity nodes can - scale to hundreds of millions and beyond
βall with - smooth, - predictable
resource usage - distributed over time.
Each CID required its own 10-20 second DHT lookup,
DHT lookup
could only handle 5-10k CIDs
5-10k
Self-hosting a lot of content and publishing yourself as a provider
becomes viable for regular users, businesses and organizations.
Resource-constrained hardware (like Raspberry Pi) can now provide hundreds of thousands of CIDs; high-capacity nodes scale to hundreds of millions and beyond.
ships
Whatβs new: Kubo v0.39 ships Provide Sweep as the default provider system.
Metanoia
Personal
10 years later
going Inter - - Personal and - Planetary
It is not semantics
we need Mutual learning Trails
TrailBlazers for Human InterIntellect Aumentation
Indy Learning commons
Symmathesy
Symmatheticist in the medium of Softwrae
Let People do it All For Themeselves
for - contextualized
t is a holistic solution which integrates web research, linked data, annotation, note-taking and knowledge organization into a Lifelong Personal Digital Archive of βborn reproducibleβ, ab initio re-purposable, and re-enactable, Research Objects.
holistic life-long digital archive
Research Objects
contextualizes semantically structured information,
contextualize
to
conplexualize
asoociative complexes
two way linked intentionally/semantically transparence
scaling findp=bility and recall
digital pensive
harvesting and contextualizing semantically structured information from Freebase as a linked data source.
contextualizing
conlexualizing
"pinned+mfs" - a combination of the pinned and mfs strategies. βΉοΈ NOTE: This is the suggested strategy for users who run without GC and don't want to provide everything in cache
from provider strategy
the provider persists its reprovide cycle state and provide queue to the datastore, and restores them on restart
restore or restsart
from - smart-resume
The legacy provider problem
The legacy provider problem: The legacy system processes CIDs one at a time, requiring a separate DHT lookup (10-20 seconds each) to find the 20 closest peers for each CID. This sequential approach typically handles less than 10,000 CID over 22h (Provide.DHT.Interval). If your node has more CIDs than can be reprovided within Provide.DHT.Interval, provider records start expiring after amino.DefaultProvideValidity, making content undiscoverable.
Provide.DHT.SweepEnabled Enables the sweep provider for efficient content announcements. When disabled, the legacy boxo/provider is used instead.
Provide Validity of 48h, and discard the record after expiration
from expiration mechanism
Provider Records
from [pointers peers](https://hyp.is/SSfiDtaeEfCReP-M16GHLQ/ipshipyard.com/blog/2025-dht-provide-sweep/}
bucket size of k = 20
from - replication factor of 20
kubo 0.39 with sweep and fast root cid
fast root CID
0.30.0
IPFS Desktop

the problem of using location addressing
the pronlem of Location Addressing =
why we must distribute the Web
if you find a problem with the internet
or uh you have some new ideas about what humanity should uh be able to do, you can just
write a protocol uh and then you implement it and if you're right uh and it works then you tell the world and
then it gets deployed and then a lot of people will use it and the world will be a better place
A Terse, Quick IPLD Primer for the Engineer
IPLD Primer
HAMT
h
explicit versioning is fragile; feature detection allows smooth growth and natural evolution.
explicit versioning is fragile
considerable effort made to keep the amount of computation required to decide if data "matches" a Schema or not to be minimal,
not turing complete
increasingly compact/high-entropy/schemafull
compact high-entropy schemafull
more redundant data (meaning they're easier to "eyeball" and understand without a schema!);
easier to "eyeball" =
and understand
without a schema
"design language" for IPLD!
design langauge for IPLD
ADLs (short for Advanced Data Layout
schemas
combination of a CID and a Path is a context-free way to reference any information that's legible via IPLD.
contxt free reference
many nodes in a graph
selectors
Paths are a 1->1 thing: they start from one position in a DAG, and get you to single destination position.
1 to 1
Data Model level (where the important semantics are!)
semantics?
IPLD takes all of the incidental choices that must be made (but don't "matter", per se)
incidental choices that must be made but do not matter pers
thesis statement
IPLD
maps and lists for building nested structures
maps and lists
The IPLD Data Model is like an "AST" for data -- but without the "S"; the Data Model is independent of syntax.
data model independent of syntax AST
z
use inbrowser.link
https://bafybeibc6bqagreyg5oggwyomlj6pxvjmv45r44b4hjufzqkd73aafck7a.ipfs.inbrowser.link/
Owner avatar js-kubo-rpc-client
https://k51qzi5uqu5div1auuxm59ygav4p7gdg9z4e9iggtu6m43rmc3xw75mczx2b7x.ipns.dweb.link/me/indy0Pad/?isPathWritable=true&path=https://bafybeibo3cfxbpled7ybk5mp6o2xdthk7af5ebxgzm7y3haemte6ntxjnm.ipfs.dweb.link/me/%F0%9F%A7%8A-me.html&isPathWritable=true
it took some time
Next Step is to see if all this will work in Agregore
That would enable anyone with IPFS Desktop and Agregore browser to be a create and share documents on the IndyWeb
We need 1s with Peergos Account to be able to share
secret links and establish trusted collaboation between poeple they know
Use IPFS and IPNS for
me/indy)Pad and documents under me
save manually on corresponding Peergos space
download it and upload it to IPFS Desktop
to get this document describing the work towards this goal
A hash array mapped trie[1] (HAMT, /ΛhΓ¦mt/) is an implementation of an associative array that combines the characteristics of a hash table and an array mapped trie.[1] It is a refined version of the more general notion of a hash tree.
gloss
allow the root to grow lazily[
lazy grow
hash array mapped trie (HAMT)
Hash Array Mapped trie
within a commercial offering or entity
commercial offering
mission to enable conversations over the worldβs knowledge,
Hypothesis mission
enable conversation over the world's knowledge to
install the Hypothesis Bookmarklet
on mobile

javascript:(function(){window.hypothesisConfig=function(){return{showHighlights:true,appType:'bookmarklet'};};var d=document,s=d.createElement('script');s.setAttribute('src','https://hypothes.is/embed.js');d.body.appendChild(s)})();
To install the Hypothesis Bookmarklet
sunset via

get help
bring their own communities.
great slogan
Bing YOur own Communities BYOC
DIYO
Hypothesis sidebar
delighted to see my annotation 5 years ago showing up here

Ever since I learned about hypothesis in 2017 I worked to follow hypothesis example of making
Reading active and social
to "lock the web open" and make Colaborative Writing and interpersonal collaboration linked to annotation possible.
Use hypothesis as a an interpersonal search engine
Able Player Fully accessible cross-browser HTML5 media player.

Able Player Demos
x
4 object pronoun from
"me" is an object pronoun
object pronoun
/hyperpost/~/indyweb/π/20/25/11/3/setup/-/indy0wiki.pad/index.html
secret.link: setup-indy0wiki.pad
secret.link: setup-indy0wiki.pad update - 2025-12-09 // new way of self-identifying documents that are not web hosed using secret links to Peergos files and auto open to view and in response to sunsetting upr via.hypothes.is
HyperPost Presentation setup via IPFS Desktop on ThinkPad still experimental but got it to work
tools for thought
and we can annotate the presentation
e cannot
within
add the ability to share openly with foll proenance and verifiable record of the sharing

todo
/hyperpost/π/π§/index.html
Peergos Secret Link to
Does not allow annotations
But 1 can add Page Notes
4<br /> hermeticism

4<br /> hermeticism
SICP
4 hermeticism
personally archive @gyuri https://bafybeicid7zd2l56yipy5nxra2qttovqqjh46x3cqu4qsmlhfevsku5lcq.ipfs.dweb.link?filename=Release%20v0.47.0%20%C2%B7%20ipfs_ipfs-desktop%20%C2%B7%20GitHub%20(30_11_2025%2012%EF%BC%9A29%EF%BC%9A13).html
and annotated it with https://hypothes.is
and the self-hosted archved page snarfed with single File has been instantly available!
A real Milestone

identifying a specific niche or shared interest that resonates with you and others,
ipfs://π§/ β/ hyperpost/~/gyuri/ποΈ/2025/12/search/Build.YOur.own.Communities
identifying a specific niche or shared interest that resonates with you and others
archived.by @gyuri ipfs://π§/ β/ hyperpost/~/gyuri/ποΈ/2025/12/search/Build.YOur.own.Communities
naming is creating
archived.by @gyuri ipfs://π§/ β/ hyperpost/~/gyuri/ποΈ/2025/12/search/Build.YOur.own.Communities
defining a clear and unique identity that attracts like-minded individuals who share your passions
archived.by @gyuri ipfs://π§/ β/ hyperpost/~/gyuri/ποΈ/2025/12/search/Build.YOur.own.Communities
identifying a specific niche or shared interest that resonates with you and others
https://bafybeifr4fk4z5vpoqrjbcgmuevpofvhaccopzltutgfe4iixqv6ozfbo4.ipfs.dweb.link/?

Dr Gyuri Lajos software developerPresentation: hyperpost.co/app/deck/slides.html#/ Video: youtu.be/XDrowxW2u1Y?t=2659
from https://hyp.is/0P36ttA6EfCi2eMjvaZA7Q/web.hypothes.is/help/annotation-basics/
Oiginaly annotated the IPFS version https://bafybeig7nrhxx3nyb5rfmuj7cfy5xbl4ldtwr57ol6lykibww625qkxnke.ipfs.dweb.link/?filename=hyperpost-0.html
the one showing should be the canonical
Instead of shipping early shipping often
or "Get IT Right First time"
Get the IT right first.
If you do that IT can self-host and self-realize the IT
as a Conversations
After all Software is a Symmathetic Conversaiton creating the Medium, the Sweet spot, just above the threshold that is trully required.
Proving that the "Worse IS Not Better"
Need the Jewel like Diamond approach
This is truly working in the Open Learning Commons
creating the Indy (Mutual) Learning Commons
The work is not only Open Souce
developing Open constructs
easy to emulate compelling to do and etend expand exapt
but it shipf with Open Sauce
i
βοΈThe most up to date version will be the canonical version that will point to the latest version using IPNS
all versions thus will be available via links
My to do list is on the annotation margins
use hypothesis search to see a reverse chronological listing of todo tasks
This annotation is at the top of that list when this annotation was made
Origo Folder for my hyperpost Peergos Account
No Groan Zome
but
Not just Converge but UpVerge in an autopoietic emregent upward spiral
Beyond all expectations
Imagined a whole new way what that leads to is beyond prior imaginings
Origo Folder for my hyperpost Peergos Account
No Graon Zome
but
/hyperpost/π/π/gyuri/do/web/snarf/π /2025/10/2/Metaphysics.of.Adjacency.html
/hyperpost/π/π/gyuri/do/web/snarf/π /2025/10/2/Metaphysics.of.Adjacency.html
/hyperpost/π/π/gyuri/do/web/snarf/π /2025/10/2/Metaphysics.of.Adjacency.html
These models are not static
what if these aren't models
but are in morphic resonance with something that is outside the brain driven by the shift of attention
Thousand Brains Theory of Intelligence
search
Reverse engineering the neocortex to revolutionize AI
/hyperpost/π/π§/0/
Origo Folder - hyperpost web directory on IPFS
localhost
should work on asus
do.how-innotation as incantation
If you can dream up and elaborate in digital form, in an info-morphic indyweb page,
your tacit intent,
what you would like to experience to happen in a named context using using the tralmarks in line intentionally transparent notation that allows one to write to think and articulate yet to be real-ized digital experiences that can transmute formulative thinking, the butterflies into caterpillars of intentional formulations that progressively able to come to a lively digital experiences lively butterflies of digitial fwork flows to engage with and experience to realize yOur intent
name it, elaborate the implicate context, formulate eht you had in mind, write to think it through to the last detailed structures and inerpretation that would be required to accomplish what you had in mind do it progressively till it fully formed explicitly, spell it out, when you complete that you have the spell to invoke it
a practical magic incantation capable of invoking the named capability, process morphic transformation presentation to be expierice, to deliver what you spelled out, need whenever you need ot for your purposes

I was closiing stale windows. and one about do.how to add favicons
I used the notation '>do.how - favicon'
then did search for annotations that match that trailmark
and low behold I have a way of realling all my annotations marked as >do-how
Geat demonstration of the power of notation and its systematic use!
a way to leverage hypothesis as a platform for solcial netowrorked search/serendioity engine
All 1 needs is to adopt the notation! for it.
Yet another exemplar how Civilization can advance through notation
do.how - favicon
question
You and 7 others reacted
especially helpful if you're self-hosting a lot of files on a regular home internet connection.
self-hosting is now practical!
Announces content at a steady pace
instead of bursts that could slow down your internet
Before, this would silently break your node's connectivity, forcing you to restart IPFS Desktop manually.
anther great advance.
Congratulation
ππππππ ππππππ
Your Node Stays Reachable After Router Restarts

share a link right after the upload finishes and it will just work.
I did notice this.
Just in the nick of time
Well worth the nearly decade long wait!
This is a total Game Changer
β‘ Share Files Instantly
Previously, after adding files to IPFS Dsktop, it could take 30+ seconds before friends could find and download them. Now, when you add files or folders, they become discoverable on the network almost immediately (typically under a second).
daily notes - 5th week of November 2025
daily notes - Β 5th week of November 2025
daily notes - 5th week of November 2025
/hyperpost/π/π§/0/
Origo Folder - hyperpost web directory on IPFS
https://k51qzi5uqu5div1auuxm59ygav4p7gdg9z4e9iggtu6m43rmc3xw75mczx2b7x.ipns.dweb.link/π/?isPathWritable=true&path=/β/hyperpost/π/π§/0/index.html
cut the middle bit out and have access to the document to read and share
π/?isPathWritable=true&path=
/hyperpost/π/π§/0/Origo Folder - hyperpost web directory on IPFS
via dweb link
root of the ipfs indy web
Origo Folder - hyperpost web directory on IPFS
Origo Folder - hyperpost web directory on IPFS
auto.inject - hypothes.is embed.js script
work on it in /hyperpost/π/π/gyuri/π/2025/11/5/

question formulation technique
resonates
we might learn to ask good question
ask good questions