When wave 𝑤 completes (Line 34), we use the global perfectcoin to retrospectively elect some process and consider its vertexin the wave’s first round as the leader of wave 𝑤 (Line 35).

In a nutshell, the idea is to interpret the DAG as a wave-by-wave protocol and try to commit a randomly chosen single leadervertex in every wave.

Our validity property requiresthat all messages broadcast by correct processes are eventually or-dered (with probability 1), whereas most Byzantine SMR protocols(i.g., [ 17, 37 , 43 ] require that in an infinite run, an infinite number ofdecisions are made, but some proposals by correct processes can beignored.

optimal qua-dratic communication

The message complexity is herebyimproved to O(n).

O(n log n)

set of each event z such that z isa self -ancestor of a round r unique famouswitness , and x is an ancestor of z but notof the self -parent of z

(d), one event (orange) is an ancestorof each of 4 intermediate events by different creators (red)

Amuch faster approach is to pick just a few events (vertices in the hashgraph),to be called witnesses, and define a witness to be famous if the hashgraphshows that most members received it fairly soon after it was created

x.famous ← UNDECIDED

If two events have the same received round, then they are sorted by theirreceived time

If Alice and Bob both have the same hashgraph, then theycan calculate a total order on the events according to any deterministic function ofthat hashgraph, and they will both get the same answer. Therefore, consensus isachieved, even without sending vote messages

When Bob sends an event to Carol, that sequence number can be calculated byCarol, so there is no need for Bob to actually send it over the internet

Only the state itself needs to bekept. It might be possible to do this every few minutes, so there will never be ahuge number of transactions and events stored. Only the consensus state itself. Ofcourse, a member is free to preserve the old events, transactions, and states, perhapsfor archive or audit purposes. But the system is still immutable and secure, evenif everyone discards that old information.

To defend against attackers whocan control the internet, there are periodic coin rounds where witnesses can votepseudorandomly. This means that even if an attacker can control all the messagesgoing over the internet to keep the votes carefully split, there is still a chancethat the community will randomly cross the 2n/3 threshold. And so agreement iseventually reached, with probability one.

The protocol might re-quire that Alice send those events in topological order, so Bob will always have anevent’s parents before receiving the event

to a random member

Issuing a Transaction: The client c1 composes and signs a transaction.Client c1 sends the transaction to the validators. If the inputs of the trans-action are not part of genesis, the client also sends the confirmations of theinputs to the validators.

Finally, the high communication complexity of the underlying protocols, i. e., consensus of shards, handling cross-shard transactions, and bootstrapping of participants to shards, can make the bandwidth become the bottleneck.

Furthermore, we show that against a somewhat adaptive adversary, sharding protocolsmust periodically compact the state updates (e. g., via checkpoints [26], cryptographic accumulators [7], zero-knowledgeproofs [6, 31] or other techniques [12, 19–21]); otherwise, the necessary bandwidth resources of each participantincrease infinitely in time, eventually exceeding the resources needed in a non-sharded blockchain

Additionally, we show that scalable sharding isimpossible if participants maintain (and verify) information on all other shards with which they execute cross-shardtransactions. This happens because the storage requirements become proportional to that of a non-sharded protocol.

To this end,Bob’s representative replica collects and aggregates CREDITmessages for the same incoming payment into a dependencycertificate for Bob’s xlog.

has messagecomplexity of O(N 2)

Astro guarantees total order within—but not across—xlogs

Essentially, preventing Alice from double-spending meanspreventing her from reusing sequence numbers. To do so, thebroadcast layer in Astro provides Byzantine resilience. Thisensures that a malicious client cannot broadcast two differentpayments with the same sequence number.

unlike consensus

Astro builds on the insight that consensus is unnecessary toprevent double-spending

Figure 1

While that prohibits applications thatneed a unique total ordering on conflicting transactions, likecryptocurrencies where double spending must be prevented

The gossip-about-gossip method allows for the history ofall the communications within an infrastructure to be reconstructed

None of these methods, though, includesdynamic membership, and this is a key differentiator in this paper

is proved in section 5 that if x and y are ondifferent branches of an illegal fork, then w can strongly see at most one of x andy, but not both.

If there are n members (n > 1), then an event w can strongly see an event x,if w can see more than 2n/3 events by different members, each of which can seex.

In other words, w can see x if x is known to it, and no forksby that creator are known to it

If Aliceis honest, she will calculate virtual votes for the virtual Bob that are honest. Evenif the real Bob is a cheater, he cannot attack Alice by making the virtual Bob voteincorrectly.

The otherancestor events (gray) are not stored in the red event, but theyare determined by all the hashes. The other self-ancestors (darkgray) are those reachable by sequences of self-parent links, and theothers (light gray) are not.

Most Byzantine fault tolerance protocols without aleader depend on members sending each other votes. So for n members to agreeon a single YES/NO question might require O(n2) voting messages to be sent overthe network, as every member tells every other member their vote. Some of theseprotocols require receipts on votes sent to everyone, making them O(n3). And theymay require multiple rounds of voting, which further increases the number of votingmessages sent.

It is sufficient to tell Carol that this event is the next one by Alice, and that itsother-parent is the third one by Bob. With appropriate compression, this can besent in very few bytes, adding only a few percent to the size of the message beingsent.

Alice will not send Carol the red event until Carol alreadyhas all its earlier ancestors

Instead of Alice signing a transaction she creates, she will sign theevent she creates to contain that transaction. Either way, she is only sending onesignature

Virtual voting - every member has a copy of the hashgraph, so Alice cancalculate what vote Bob would have sent her, if they had been runninga traditional Byzantine agreement protocol that involved sending votes.So Bob doesn’t need to actually her the vote.

Fairness - it should be difficult for a small group of attackers to unfairlyinfluence the order of transactions that is chosen as the consensus.

An algorithm for a single YES/NO decision can then be extended to decidinga total order on a set of transactions, which may further increase the vote traffic.

A proof-of-expired-timesystem [9] can avoid the wasted electricity (though perhaps not the cost of miningrigs) by using trusted hardware chips that delay for long periods, as if they weredoing proof-of-work computations.

It also means that itis necessary to slow down how fast blocks are mined, so that the community canjointly prune branches faster than new branches sprout. That is the purpose of theproof-of-work. By requiring that the miners solve difficult computation problemsto mine a block, it can ensure that the entire network will have sufficiently longdelays between mining events, on average

though at any time, if an honest memberrepeatedly sends messages to another member, the attackers must eventually allowone through

Example 9.3 (Bait-and-Switch Attack).

In the permissioned setting there is the following inter-esting line of research. The aim is to construct an atomicbroadcast protocol based on a combined encoding of thedata transmission history and voting on “leader blocks”.Such protocols allow the network participants to reach aconsensus on a total ordering of the received transactions,and this linearised output forms the ledger.

The main difference of our proposal to all the aforemen-tioned protocols is that consensus is found on the heaviestDAG without the need for a “linearisation” using any leaderselection. This reduces the purposes of blocks to data trans-mission and voting

These protocolsremove the bottleneck of data dissemination of the classicalNakamoto consensus by decoupling the data disseminationfrom the consensus finding

Blockchain-based protocols relyon a chain or “linearisation” of blocks that create a totalorder of transactions. These blocks have three purposes:leader election, data transmission and voting for ancestorblocks through the chain structure, see [34]. Each of theseaspects can be, individually or combined, addressed througha DAG-based component. The various proposals differ inhow and which of these components are replaced by a DAG.

Tangle 2.0Leaderless Nakamoto Consensus on the Heaviest DAG

However, as soon as a node from X changesits vote, the resulting situation is symmetric to the initialcondition

Upon receipt of the missing parent block,the past cone is now complete (unless their parents aremissing - in which case the node has to repeat this procedurerecursively).

Improvements are proposed,for example, in Hashgraph [41] and Aleph [42] and morerecently in Narwhal [43] based on the encoding of the “com-munication history” in the form of a DAG

Probabilistic. The units appended to the system are alwaysaccompanied by the risk of being reversed. The probability isinversely proportional to its cumulative confidence (includ-ing the forms of depth/score/weight etc). The confidence isobtained from the contributions of subsequent units.- Deterministic. The units, once attached to the system atsome point in time, will instantly and permanently get con-firmed so that they can never be removed from the system.

5.1 Properties Between BA and NC

Secondly, building smartcontracts or state transition applications rely on linear order

Each partici-pated node maintains a separate chain, and nodes mutually interactvia the gossip protocol. The node locally creates an event to recordthe history of received information

Graphchain, instead, introduces an incentivemechanism to maintain the graph. Transactions are required topost transaction fees as the offering for collection. That means eachtransaction must refer to a group of ancestor transactions to de-plete their deposit (fees) when verifying the validity. Meanwhile,ancestor transactions must have enough fees for collection. Feesare depleted by incoming transactions from the oldest ancestorsin paths to reach a prescribed total amount. High-fee transactionsattract powerful miners working in parallel for rapid confirmation,while low-fee transactions will finally be picked up by a small miner.Based on this design, Graphchain makes the current transactionsquickly become enshrined in the ancestry of all future transactions.

Specifically,the extension of the tangle follows the rule3 where one tip (pend-ing/unapproved transaction) is required to approve two ancestortransactions. Thus, users who issue a transaction will contributeto the security of the system.

several systems,both tie-breaking and conflict solving are integrated intothe extension rule. The consensus mechanism, by this way,reaches the agreement in one-step

Unlike the alternatives, HoneyBadgerBFTsimply does not care about the underlying network

A program withnon-monotonicity can be made consistent by including coordinationlogic at its points of order.

The most telling example fromSection 3.2 is the use of tombstoning for low-latency deletion. Inpractice, memory for tombstoned items must be reclaimed, so even-tually all machines need to agree to delete some items.

A related pattern in storage systems is the use of tombstones: specialdata values that mark a data item as deleted. Instead of explicitlyallowing deletion (a non-monotonic construct), tombstones maskedimmutable values with corresponding immutable tombstone val-ues.

CALM provides the formal framework forthe widespread intuition that we can indeed “work around CAP”in many cases, even if we violate traditional systems-level notionsof storage consistency.

This is the conclusion of the Dynamo design.In later work [18 ], we go further to make checkout monotonic inthis setting as well. : the checkout operation is enhanced with amanifest from the client of all its update message IDs that precededthe checkout message: replicas can delay processing of the checkoutmessage until they have processed all updates in the manifest

A keyinvariant of these coherence protocols is thateither 1) a cache line is not present in anycache, 2) a mutable copyis present in a single cache,or 3) the line is present in anynumber of caches but is immutable

At the core of this dissertation’s approach is this scalable commutativityrule: In anysituation where several operations commute—meaning there’s no wayto distinguish theirexecution order using the interface—theyhave a implementation that is conflict-free duringthoseoperations—meaning no corewrites acachelinethat was reador written byanother core.Empirically, conflict-free operations scale, so this implementation scales. Or, more concisely,whenever interface operations commute, theycan be implemented in a waythat scales.

Importantly, the external transaction determines valid modifiers of each state.

It isinflexible for a node to join or leave a workflow at any time since the config-uration of channels is fixed.

Moreover, interactions between channels rely on atrusted channel [3] or an atomic commit protocol [2], which either breaks thedecentralization or still treat the system as an entirety.

The configuration of a channel may be updated using achannel configuration update transaction. This transactioncontains a representation of the changes to be made to theconfiguration, as well as a set of signatures.

The relative ordering of aborted trans-actions is kept the same. As a result, every transaction cancommit eventually, since the first transaction in a batch al-ways commits and the position of an aborted transactionalways moves forward across batches

However, as the orderingbetween transactions is determined by consensus and fixedacross all nodes, we leverage this to permit both transactionsto write to different copies of the object, but subsequentlycommit only the first as determined by the ordering.

Hot Key Theorem. Let l be the average time between atransaction’s execution and its state transition commitment.Then the average effective throughput for all transactionsoperating on the same key is at most 1l .

Compatibility with external oracles: To allow the useof external oracles in the deterministic second executionphase, we gather and save oracle inputs in the pre-orderexecution step.

A dependency exists when two transactionsoverlap in some keys of their RW sets (read-only transactionsdo not even enter the orderer). In that case, we cannot processthem independently. Therefore, we need to keep track ofdependencies between transactions so we know which subsetsof transactions can be processed concurrently

This requires theuse of a deterministic contract language, such as Ethereum’sSolidity, which must be learned by the application developercommunity

With this optimization, TAPIR can now accept: (1) readsof past versions, as long as the read timestamp precedes thewrite timestamp of the next version, and (2) writes in thepast (per the Thomas Write Rule [45]), as long as the writetimestamp follows the read timestamp of the previous versionand precedes the write timestamp of the next version.

A peer checks if a sufficient number of valid endorsingpeer signatures, based on the endorsement policy, have been col-lected (Validation System Chaincode (VSCC) validation)

Table 1: Notations for sets

Optionally, the clientmay check the validity of the endorsing peers signatures and theconsistency between the read/write set received from differentpeers.

The client collects the endorsing peers responses andsends them to the ordering service nodes

First, wedeliberately do not send transaction read-write sets to theorderers, only transaction IDs. Second, we chose to keep toFabric’s design goal of allocating different tasks to differenttypes of nodes, so our orderers do not examine the contents ofthe read and write sets

Arguably, the use of BFT protocols in permissionedblockchains is not as important as in permissionless sys-tems because all participants are known and incentivizedto keep the system running in an honest manner.

(a) to achieve con-sensus on the transaction order

Transactions follow an execute-order-commit flowpattern instead of the common order-execute-commit pattern.Client transactions are first executed in a sandbox to determinetheir read-write sets, i.e., the set of keys read by and writtento by the transaction. Transactions are then ordered by anordering service, and finally validated and committed to theblockchain

Furthermore, it doesn’t say that every reorderinghas indistinguishable results on a given implementation;it requires instead that every reordering is allowed bythe specification to have indistinguishable results.

But as noted above, a partial order like happens-before is iso-morphic to a join semi-lattice! That means that we can capture theCmRDT log as a CvRDT. One simple way to do this is to hold theDAG corresponding to the partial order: this is simply a “grow-only”set—whose only operation is add—which holds edges of a DAG.Each directed edge connects two operations, and indicates that theoperations happened in the order of the edge. (Another solutionwould be to use CvRDTs for vector clocks.) A natural query oversuch a CvRDT is to “play the log”: i.e. produce a topological sort ofthe log and apply the operations in the resulting order.

Instead of explicitly allowing deletion (a non-monotonic construct), tombstones mask immutable values with corresponding immutable tombstone values.

deletion (a non-monotonic construct)

Unfortunately, neither the add nor delete operation commutes with checkout—if a checkout message arrives before some insertions into either the Added or Deleted sets, those insertions will be lost.

As a traditional approach to avoid such "race conditions," we might bracket every non-monotonic delete operation with a global coordination to agree on which add requests come before it. Can we do better?

However, this powerful abstractioncomes with a hefty cost: concurrent transactions must coordinate inorder to prevent read/write conflicts that could compromise equiv-alence to a serial execution.

we only consider a singleset of invariants per database rather than per-operation invariants

we explic-itly require more information from the application in the form ofinvariants (Kung and Papadimitriou [38] suggest this is informationis required for general-purpose non-serializable yet safe execution.)

However, we are not aware of related work that dis-cusses when coordination is strictly required to enforce a given set ofinvariants

or safety (e.g., the CRDT OR-Set does not—by itself—enforce invariants, such as ensuring that no employee belongs totwo departments), and are therefore not sufficient to guarantee cor-rectness for all applications

This means that, inour payroll example, we can provide coordination-free support forconcurrent salary increments but not concurrent salary decrements

To avoid variants of these anomalies, many optimistic, coordination-free database designs have proposed the use of abstract data types(ADTs), providing merge functions for a variety of uses such ascounters, sets, and maps [19, 44, 50, 58] that ensure that all updatesare reflected in final database state.

However, the reliance on invariants also has drawbacks. I-confluence analysis only guards against violations of any providedinvariants. If invariants are incorrectly or incompletely specified, anI-confluent database system may violate application-level correct-ness.

If two indepen-dent decisions to commit can result in invalid converged state, thenreplicas must coordinate in order to ensure that only one of the deci-sions is to commit.

Theorem 1 establishes I-confluence as a necessary and sufficientcondition for invariant-preserving, coordination-free execution

A system is globally I-valid iff all replicas alwayscontain I-valid state

A system is convergent iff, for each pair of servers,in the absence of new writes to the servers and in the absence ofindefinite communication delays between the servers, the serverseventually contain the same versions for any item they both store

invariants, orpredicates over replica state: I : D →{true,f alse}

On a wide-area network, delay is lower-bounded bythe speed of light (worst-case on Earth, around 75ms, or about 13operations per second [9]). Under network partitions [13], as delaytends towards infinity, these penalties lead to unavailability [9, 28]

The classic answer to maintain-ing application-level invariants is to use serializable isolation: ex-ecute each user’s ordered sequence of operations, or transactions,such that the end result is equivalent to some sequential execu-tion [15, 30, 53].

Given knowledge of application transactions and correctness crite-ria (e.g., invariants), it is often possible to avoid this coordination(by executing some transactions without coordination, thus pro-viding availability, low latency, and excellent scalability) whilestill preserving those correctness criteria.3. Invariant confluence offers a necessary and sufficient conditionfor this correctness-preserving, coordination-free execution.

Serializable transactions preserve application correctness at thecost of always coordinating between conflicting reads and writes.

In this paper, we address the central question inherent in this trade-off: when is coordination strictly necessary to maintain application-level consistency? To do so, we enlist the aid of application pro-grammers to specify their correctness criteria in the form of invari-ants.

Use cases that requiretransactional isolation of repeatable reads [ 5] are not a good fit,as FabricCRDT commits transactions even if their read-set is out-dated.

According to our findings, FabricCRDTsuccessfully merges all conflicting transactions without any failureswhen all transactions use CRDTs.

CRDT Transactions in a Block

Once a transaction fails, the only option for clients is to create anew transaction and resubmit, which adds to the complexity ofFabric application development

where there exists a happened-before relationVoteRegister1

Rather than moving forward by a fixed number of predecessors in each round trip, this algorithm allows some commits to be skipped, so that it reaches the common predecessors faster.

Instead of resolvingbranches, it permits them, resulting in a Directed AcyclicGraph (DAG) structure of the blockchain rather than a linearone.

In this chapter, we address theorthogonal problem of automatically suggesting an efficient set of MV for a given workloadof SPARQL queries.

Getting the complete set of solutions of q would require to query the residual pattern(a?,p4,?e) over the RDF dataset and join the results with the partial results of i1 andi2.

We focus on covers with overlapping embeddings since they allow better estimations ofthe cost savings that can be achieved with them (see Section 2.3).

Query

However, for the rest of this thesis we only considerintensional overlaps to avoid the costly pre-computation and maintenance of extensionaloverlaps

Let G be a data graph, I the set of indexes on G, andQ a query against G. We call an index I ∈ I eligible for Q iff P(I) v P(Q). The set ofall eligible indexes for query Q is denoted by IQ.

itis not possible to map variables ?y and z from i2 to any variable of q

each occurrence of the triple pattern (?a,p1,?b) of q must be contained inthe set of occurrences of i1

During index creation, indexes are computed andpersistently materialized and the results are made available for the query processor. Inour approach, indexes are defined offline, i.e., created by an administrator before queriesare executed.

The frequency of I in G, written as #G(P), is thenumber of occurrences of P in G.

O represents the set of all occurrences of P in G

which is the set of triple patterns in the body of Q

Figure2.1 shows the graph representation of a triple pattern.

The query pattern must be divided into that part that is executed byusing materialized results and the rest of the query patterns that are not possible or notbeneficial to execute using them

1.2.6 Materialized View Selection Problem

The MV–selection problem can be viewed as a combinatorial optimization problemsince its solution has to be chosen from among a finite number of possible configurations.Therefore, explicit and complete enumeration and scoring of all possible MV subsets is,in principle, a possible way to solve it. This method is however impractical in most cases,since the computational effort grows exponentially with the number of candidate MV[25].

However, we deal with the fact that queries may notbe fully answered using only materialized results. Thus, the partial results need to beextended to answer the query.

Partial Materialized Views

doc.get(“todo”).idx(1).get(“done”) := true;

In order to generate globally unique operation identifierswithout requiring synchronous coordination between repli-cas we use Lamport timestamps [42].

“done”: true

Agregore works by letting you load web pages and content from peer to peer protocols the same way you would load them from a website.

Append a new value to the autobase.

Hypercore is a secure, distributed append-only log built for sharing large datasets and streams of real-time data. It comes with a secure transport protocol, making it easy to build fast and scalable peer-to-peer applications.

Multi-writer will allow Dats to be modified by multiple devices and multiple authors at the same time. Each author will have their own secret key and publish a Dat with their data in it. Multi-writer fuses all of these separate Dats into one “meta-Dat” that is a view of everyone’s data combined

multi-causal operations will prevent the structured log from being linearly interpretable and may severely affect performance, simplicity, and intent

This won’t be convergent by default since the operations don’t have an inherent total order, but it’s easy to fix this by giving each one a globally-unique ID in the form of an owner UUID 3 plus a Lamport timestamp. (In the diagrams below, this is encoded as “SX@TY”, where “SX” represents the UUID for site X, and Y is just the timestamp.) With this scheme, no two operations can have the same ID: operations from the same owner will have different timestamps, while operations from different owners will have different UUIDs

Nonetheless, I made one additional tweak to ensure that remapping only happens very rarely. Instead of storing just the UUID in the site map, I also store the wall clock time at which the UUID was added. In the site map, these tuples are sorted first by time, then by UUID. Assuming that modern connected devices tend to have relatively accurate clocks (but not relying on this fact for correctness), we can ensure that new sites almost always get appended to the end of the ordered array and thus avoid shifting any of the existing UUIDs out of their previous spots. The only exception is when multiple sites happen to be added concurrently or when the wall clock on a site is significantly off.

I’ve solved this with the help of a secondary CRDT that’s stored and transferred along with the CT: an ordered, insert-only array of known UUIDs called the site map. The 16-bit site identifier corresponding to a UUID is simply its index in the array.

We see further interesting problems around types, schemamigrations, and compatibility. Different collaborators may beusing different versions of an application, potentially withdifferent features. As there is no central database server,there is no authoritative “current” schema for the data. Howcan we write software so that varying application versionscan safely interoperate, even as data formats evolve?

CRDTs do not require a peer-to-peer networking layer;using a server for communication is fine for CRDTs. However,to fully realize the longevity goal of local-first software, wewant applications to outlive any backend services managedby their vendors, so a decentralized solution is the logicalend goal.

URLs are a good mechanism for sharing

Visualizing document history is important

However, in all the prototypes we devel-oped, we found that the default merge semantics to be suffi-cient, and we have so far not identified any case requiringcustomised semantics

In practice, the CouchDB model has not been widelyadopted [42]. Various reasons have been cited for this: scala-bility problems when a separate database per user is required;difficulty embedding the JavaScript client in native apps oniOS and Android; the problem of conflict resolution; theunfamiliar MapReduce model for performing queries; andmore. All in all, while we agree with much of the philosophybehind CouchDB, we feel that the implementation has notbeen able to realize the local-first vision in practice.

These thick-client apps have the advantage of being fastand working offline, because the server sync happens in thebackground.

web apps will never be ableto provide all the local-first properties we are looking for,due to the fundamental thin-client nature of the platform

In many web browsers,if the user clears their cookies, all data in local storage isalso deleted [121]; while this is not a problem for a cache,it makes the browser’s local storage unsuitable for storingdata of any long-term importance

In principle it is possible to collaborate without a repository service, e.g.by sending patch files by email [48],

It’s interesting to note that most software engineers havebeen reluctant to embrace cloud software for their editors,IDEs, runtime environments, and build tools. In theory, wemight expect this demographic of sophisticated users to em-brace newer technologies sooner than other types of users.

But Git has no capability forreal-time, fine-grained collaboration, such as the auto-matic, instantaneous merging that occurs in tools likeGoogle Docs, Trello, and Figma

Git is excellent for asynchronous collaboration, espe-cially using pull requests (Figure 6, [61]), which takea coarse-grained set of changes and allow them to bediscussed and amended before merging them into theshared master branch

If your computer’s hard drive fails, you canrestore your work simply by installing the app and waitingfor it to sync.

The flip side to this is a total loss of ownership and control:the data on the server is what counts, and any data on yourclient device is unimportant — it is merely a cache

Attachments are easy to understandand trustworthy.

Using a version control system such as Git

Consider a significant personal creation, such as aPhD thesis or the raw footage of a film. For these you mightbe willing to take responsibility for storage and backups inorder to be certain that your data is safe and fully under yourcontrol

With data ownership comes responsibility: maintainingbackups or other preventative measures against data loss,protecting against ransomware, and general organizing andmanaging of file archives

so you have the freedom to process thisdata in arbitrary ways

You should be able to copy and modifydata in any way, write down any thought,7 and no companyshould restrict what you are allowed to do

Although there does not seem to be a great danger ofGoogle shutting down Google Docs anytime soon, popu-lar products (e.g. Google Reader) do sometimes get shutdown [106] or lose data [105], so we know to be careful

Cuneiform script on clay tablet, ca. 3000 BCE. Image from Wikimedia Commons [5].

Besides having severalpeople edit the same document in real-time, it is sometimesuseful for one person to tentatively propose changes thatcan be reviewed and selectively applied by someone else

neverneeding to show you a spinner while you wait

Local-first software is different: because it keeps the pri-mary copy of the data on the local device, there is never aneed for the user to wait for a request to a server to complete.

The user interface may try to hide that latency by showingthe operation as if it were complete, even though the requestis still in progress — a pattern known as Optimistic UI [92] —but until the request is complete, there is always the possibil-ity that it may fail (for example, due to an unstable Internetconnection).

Online eventprocessing (OLEP) [36] is an approach for handling such multi-partition interactions by breaking them down into multiple streamprocessing stages.

then it is no longer sufficientto process the events in each partition independently

A challenge in many event-based systems is how to handle changesin the schema or data format [56]. The problem is especially pro-nounced in systems where we cannot guarantee that all replicasare running the same version of the software. For example, in ap-plications that run on end-users’ devices, it is up to the user todecide when to install a software update; thus, a user who is hes-itant to install updates may be running a much older version ofan application than a user who always installs the latest version.Nevertheless, those users should be able to interoperate as much aspossible, which means that any data format changes must be bothforward and backward compatible. The challenge becomes evengreater if users are able to customise the software they are running,e.g. through end-user programming [30].

In systems that are based on immutable events, one promisingapproach is to use bidirectional functions (lenses) to convert be-tween different versions of a data model, which allows differentreplicas to use different state representations while still being ableto interoperate [49].

Partially ordered event-based systems are well placed to supportsuch branching-and-merging workflows, since they already makedata changes explicit in the form of events, and their support forconcurrent updates allows several versions of a dataset to coexistside-by-side.

in Git terms, one user can create a branch (a setof commits that are not yet part of the main document version),and another user can choose whether to merge it

Database transactions support a weak form of multi-version con-currency control by allowing an uncommitted transaction’s writesto be either committed or rolled back by aborting the transaction [8].However, most databases do not allow one user to share the state ofan uncommitted transaction with another user, and most databasesdo not allow the user to find out what data has changed in anuncommitted transaction (the equivalent of git diff). Moreover,in most database systems, an uncommitted transaction may holdlocks and thus prevent other transactions from making progress.

compute the differencesbetween versions of the document to visualise the change history

we can reconstruct the state of thedocument at any past moment in time

an initial event represents only the intention toperform a certain action

Moreover, if processing an event may have external side-effectsbesides updating a replica state – for example, if it may trigger anemail to be sent – then the time warp approach requires some wayof undoing or compensating for those side-effects in the case wherea previously processed event is affected by a late-arriving eventwith an earlier timestamp. It is not possible to un-send an emailonce it has been sent, but it is possible to send a follow-up emailwith a correction, if necessary. If the possibility of such correctionsis unacceptable, optimistic replication cannot be used, and SMR oranother strongly consistent approach must be used instead. In manybusiness systems, corrections or apologies arise from the regularcourse of business anyway [27], so maybe occasional correctionsdue to out-of-order events are also acceptable in practice.

However, when a replicaprocesses events out of timestamp order (inserting an event some-where in the middle of the timestamp-ordered sequence), it mustbe able to roll back the replica state to the state at the time corre-sponding to the insertion position, apply the new event, and thenreplay the events whose timestamps are greater than that of thenew event [64]. This approach is known as time warp [31].

In a partially ordered system it is still possible to enforce a to-tal order on events after the fact, as illustrated in Figure 2. Wedo this by attaching a logical timestamp to each event; Lamporttimestamps [45] are a common choice.

However, other eventsmay be concurrent, which means that neither happened before theother; in this case, different replicas may process those events in adifferent order [10].

If permanent deletion of records is required (e.g. to delete per-sonal data in compliance with the GDPR right to be forgotten [62]),an immutable event log requires extra care.

Disadvantages of the OLEP ap-proach. In the previous examples, log consumers update the state in data stores (the database and search index in Figure 2; the account balances and account statements in Figure 3). While the OLEP approach ensures every event in the log will eventually be processed by every consumer, even in the face of crashes, there is no upper bound on the time until an event is processed.This means if a client reads from two different data stores that are up-dated by two different consumers or log partitions, then the values read by the client may be inconsistent with each other. For example, reading the source and destination accounts of a payment may return the source ac-count after the payment has been pro-cessed, but the destination account before it has been processed. Thus, even though the accounts will even-tually converge toward a consistent state, they may be inconsistent when read at one particular point in time

Splitting a “transaction” into a mul-tistage pipeline of stream processors allows each stage to make progress based only on local data; it ensures that one partition is never blocked waiting for communication or coordination with another partition.

From a data analysis point of view, an event log is more valuable than the state in a database. For example, in an e-commerce setting, it is valuable for business analysts to see not only the final state of the cart at checkout, but also the full sequence of items added to and removed from the cart, since the removed items carry information, too (for example, one product is a sub-stitute for another, or the customer may return to buy a certain item on a later occasion).

In contrast, in a database that sup-ports arbitrary insertions, updates, and deletes, it is much harder to re-cover from incorrect writes, poten-tially requiring the database to be re-stored from a backup.

Some log-based stream processors such as Apache Flink support so-called exactly-once semantics, which means that even though an event may be pro-cessed more than once, the effect of the processing will be the same as if it had been processed exactly once. This behavior is implemented by manag-ing side effects within the processing framework and atomically commit-ting these side effects together with the checkpoint that marks a section of the log as processed.

Since it is possible for an event to be processed more than once when re-covering from a failure, state updates must also be idempotent.

Thus, if a subscriber crashes and re-starts, it may append duplicate events to other logs.

A subscriber periodically check-points the latest LSN it has processed to stable storage. When a subscriber crashes, upon recovery it resumes pro-cessing from the latest checkpointed LSN. Thus, a subscriber may process some events twice (those processed between the last checkpoint and the crash), but it never skips any events. Events in the log are processed at least once by each subscriber.