Learn Podcast RCast

RCast 42: Interoperability and Cooperation

Subscribe to RCast on iTunes | Google Play | Stitcher

Greg Meredith discusses the state of blockchain with Isaac DeFrain and Christian Williams.


Greg: Today, I wanted to talk a little bit about some of the other blockchains. I’m not interested in criticizing other blockchains or offerings. Someone from our community asked if I would do a competitive analysis. I’ve said the same thing over and over again since I got into the blockchain space: I don’t really think that the blockchain is about competition. I get a feeling from a lot of people, especially Vitalik and Vlad and others, that it’s mostly about providing an alternative infrastructure. We’re all working together to provide that alternative infrastructure. That’s why there was so much collaboration between Ethereum and RChain. We were interested in finding a consensus algorithm that worked. 

It isn’t about competition. I know that it’s very difficult for people in the financial space to not have a scarcity mindset rather than having an abundance mindset. The one thing I can tell you is that we have an abundant supply of work. There are endless amounts of work that we have to do if we actually want to take seriously dealing with the consequences of how the climate is changing. Beyond that, there are all kinds of work. One of my favorite sayings is “Art is long, life is short.” There’s an endless amount of really fun work related to the math and the technical aspects, let alone some of the economic aspects. 

There are a lot of projects out there focusing on interoperability between networks. A good example is Polkadot. Their claim to fame is interoperability. From the beginning, RChain has been about interoperability. Part of the reason that I emphasize composition and compositionality as features is not just because that’s the only way to scale and that’s how nature scales. If you look at large-scale engineering systems, you have to build them out of components that you compose together into solutions. Human attempts to scale the way nature scales; it’s all about composition. 

If you look carefully at our model for composition, it also provides a model for interoperability. I want to talk a little bit about that from the perspective of two kinds of issues that you face with respect to trust, and then move on from there to other kinds of security guarantees. 

Let’s start with the sharding of a single chain. What we’re suggesting is that there’s a collection of—I’ll call them validators. I’m not trying to be too Casper-specific, although that’s the terminology we use with Casper. There are some validators, some computational elements that are providing the consensus mechanism. Then we can imagine another group of validators that are also providing a consensus mechanism. We want to be able to have them compose. That kind of composition is just interoperability, but with some added assumptions about the homogeneity potentially of the consensus algorithm. In other words, the consensus algorithm being used in Shard A is the same as the consensus algorithm being used in Shard B.

Isaac: When you’re talking about the validators from either of these two sets composing, what exactly are they composing there? Is it the consensus algorithm that they’re using?

Greg: Ultimately, the composition needs to be able to provide a mechanism by which you can have compute that involves actions in both chains. For example, a transaction that involves resources in both chains. That will be a consensus about the resources in A and a consensus about the resources in B, and then some higher-level consensus that governs both A and B. 

Christian: This is across the sharding? 

Greg: Yes, exactly. Does that make sense? 

Isaac: Yeah, definitely. 

Greg: One of the nice things about the way RChain does things is that we can simply assume that we have a bifurcated namespace. Shard A is operating a Namespace A, Shard B is operating a Namespace B. When we want to combine them, what we’re talking about as a composite namespace. That would be sufficient for a wide range of problems. However, there are some additional trust issues that have to be developed. 

If we just look at it from a purely theoretical point of view, just being able to bifurcate the namespace and then recompose the namespace is a great solution. By the way, that also works for interoperability. If we say that Ethereum addresses constitute a namespace or bitcoin addresses constitute a namespace, if you have a solution that allows you to have heterogeneous consensus algorithms, then the namespace approach solves a big chunk of the interoperability problem. 

We’ve known this for a very long time. URIs and URLs represent a namespace-based approach to accessing resources on the internet. A lot of what happens in blockchain is that we forget all the lessons that we’ve learned on the internet.

One lesson that we’ve learned is having composable name-based access to resources is an important piece of the puzzle. Over the last two decades, from an economic perspective, the internet has all been about the growth of the digital asset markets. Whether you’re talking Github or Spotify or Facebook or Instagram or even digital finance, it’s all been about digital assets. That’s where all of the growth is.

It’s really straightforward. Data is king in the story. If you can’t handle large chunks of digital assets, you’re not going to compete. Somehow the blockchain markets forget all of these basic lessons that we’ve learned from the internet. 

Back to sharding. One of the things that’s really key in that approach is that our approach to sharding is an approach to interoperability under the assumption that you can handle a heterogeneous consensus.

Isaac: The way that you’re saying the interoperability could work using this namespace sharding is basically treating all bitcoin addresses as certain names in a specific namespace and all Ethereum addresses as names in a different namespace. Is that what you’re saying?

Greg: That’s exactly what I’m saying. The next piece of the puzzle that you have to have: even in the homogenous consensus setting, there’s another thing that you have to be aware of. You could imagine that a shard, they all collude to attack other shards. They’re all saying these are the transactions and resources that we’ve validated so far. They’re doing that in order to hide a double spend and thereby take advantage of the other shards in the network. 

They only have to do this for a very short period of time. It doesn’t have to be a long-range attack in that case. That’s where things get really interesting because you can’t assume, even in the homogeneous case, the same kind of trust model if you’re just taking the namespace approach or any other approach. You have to have some mitigation of that risk. 

The issue is: how do you mitigate that risk? The solution that RChain proposes—a lot of this came from discussions that we had with Nash and Mike and Vlad—I really like Nash’s metaphor of the UNIX file system.

Christian: Each shard is in some sense its own chain.

Greg: That’s right. 

Christian: Putting a structure on the set of shards is somehow also putting the structure on the hierarchy of chains that you’re talking about?

Greg: That’s exactly right. At the relationship between parent and child, you’ve set up a double-sided contract, which is effectively saying that the parent’s tokens are backing the child’s tokens. It effectively gives you exchange between the resources at the parent and the child. That ameliorates the risk because that contract, and stitching together all those contracts in this tree, the parent is watching all of the children. The parent is effectively responsible for any failure of the children and so on up. People interested in the security of the network become incentivized to climb higher into the tree in order to gain more security. They are incentivized to move lower in the tree in order to gain speed. 

Christian: That makes sense. 

Greg: We’re talking a fairly high level because it’s very difficult to compare things at a low level, but if you start going at a low level, that’s where things really make a difference. I think Gavin is an amazing engineer. I was really impressed by the Whisper gossip network and other work that Gavin has done. I’m not in any way trying to slag Polkadot or any other project. That’s not my aim. As an engineer, I’m trying to compare things and what we know with math. 

But if you go to the Polkadot website and their literature, they talk about an aggregation of state machines. The whole reason, the raison d’être, for the invention of CCS and the Pi Calculus and that whole area of work, is because state machines don’t compose. In particular, they don’t compose along the lines of machine and environment. Even Mealy and Moore machines don’t compose in that way. That was essentially why Milner invented CCS. 

As I learned the hard way, when you try to compose state machines in the way that they do compose, then you get an exponential blow-up. It’s very, very, very fast and very bad. What you end up having to do is to model the constraints on the product space of the states. The complexity of that goes through the roof. 

One of the things that got me into the Pi calculus and the process calculi, in general, was doing work on relax transaction processing. Back in the late eighties/early nineties, I was working with a team on the area of what’s called relax transaction processing. This is more sophisticated than Elliot Moss-style nested transactions. 

There were a lot of different proposals for these kinds of relax transaction models, the sort of standard cases when you’re going on a trip and you need to book your hotel and your flight and your transportation. It’s okay if the transport doesn’t go through, but it’s not okay if either the flight or the hotel don’t go through, or some combination thereof. It’s not like a nested transaction where it’s all or nothing. 

We were looking at composed state machines as a way of approaching this. It was for similar reasons. These are autonomous networks. The hotel network is autonomous from the flight network, which is autonomous from the transport network. You need to get commitment across all of these networks in order to have a successful trip. 

When we were modeling the composite state machines—we really did model them; we really did have them do the product space, a state-space kind of approach—that will just bring a computer to its knees. Even with today’s compute power, it will bring a computer to its knees in a very short period. In other words, it’s not scalable. 

Once you start looking at composing state machines so that an environment is also a state machine, then you have to deal with the polarity—what’s input and what’s output. As soon as you have this polarity, what’s input and what’s output between the two different machines, that’s how the Pi calculus arises: under the assumption that your network topology is dynamic. You get a value-passing CCS if your network topology remains constant. If you can—as you can on the internet today—send a new URL to another agent, then you are already in Pi calculus territory. 

Christian: This is why I’ve always been amazed that the Pi calculus hasn’t already taken over the world is because it’s not just the language of computers. It’s the language of the internet. 

Greg: That’s precisely it. The other piece of it is, as we’ve explored over the last several podcasts, we get to reason about these things equationally. I went through the Polkadot white paper and I didn’t see any equational reasoning. It’s pages and pages long and there isn’t a single line of equational reasoning. The Rho calculus, which fixes the namespace issues in the Pi calculus, can be written down on a single page.

Whether we’re talking about Rholang 1.1 semantics, where I proposed these language features, I propose their semantics, and then I show equationally exactly what they do and why that behavior is the desired behavior for the feature. We have a spec; it’s up on the web. You could read it. Anyone who’s a developer or engineer can go through those equations as if they were code. That’s on the smart contracting side. 

Over on the consensus side, the square root thing was entirely equational. We derived equationally a notion of fault tolerance and a leaderless, distributed consensus algorithm. Or the error-correcting codes, that again is done through equational reasoning. It’s something that we can verify in a few lines. It’s only then that we have a ghost of a chance of being able to apply formal verification to either the core protocol or more importantly, and this is where the rubber really meets the road, user-defined smart contracts. 

At the end of the day, the biggest population of code is going to come from user-defined smart contracts. That will dominate as your sources of error. You could have angels from heaven write your core protocol and it doesn’t matter. All your security exposures is with those lousy humans. 

Christian: What if you just ask everybody to double-check their code first?

Greg: The DAO code was audited and they didn’t catch the bugs. The same is going to be true over and over again. You can have really crack developers and really crack auditors and you’re just going to miss things. It’s the nature of systems of this level of complexity. You have to have support to catch these errors. The more the complexity rises—as you start adding heterogeneous consensus algorithms—then you have to have equational means of reasoning about how the different consensus algorithms compose so that you get end-to-end notions of consensus, and you have to have behavioral types that allow you to see at least to some approximation the behavior of the various contracts that are running within these heterogeneous environments.

The nice thing is that a behavioral type can abstract away the behavior of a contract that’s running in a remote chain. One of the things that I have long proposed—since 1996; no earlier than that, 1993—when I was doing a research stint at British Telecoms, I proposed to use behavioral types as a discovery mechanism. A service can say, “I need a service that meets the following type.” Using a form of residuation, which you can think of as like a kind of a division operator in logic, you can go and find things that meet a spec up to a delta, and then go and find things that will fill the delta, and you can do this recursively. Using behavioral types you can layer over a heterogeneous system of networks a discovery mechanism to stitch together contracts that will provide a service. This is what I would like AI to do for me. I would like AI to write code by assembling code out of the chunks of code that are out there.

Isaac: Just to clarify, you’re saying you have some sort of a spec for a service in mind, and then you can search for existing contracts that you may be able to put them together in a particular arrangement to fulfill this spec.

Greg: That’s exactly right. I’ve talked about this before, but it’s worth restating. If you think about a model checker or a type checker and turn it on its side, it becomes a query engine, in the following sense. I’ve got all these programs up in some data source, like on Github, for example. They’re in a language that I know how to model check or type check. So I send in a spec, which is a particular property or a particular type. Then I go and brute force check every single program in the repo against this property. The ones that satisfy the property, those are the ones that I return as the response to the query.

Isaac: You just filter the database by this particular property…

Greg: That’s exactly right. The first refinement of this idea is to recognize that type systems don’t give us a lot of useful searches. Mike Stay has a really good example of this. If you look at ordinary functional types, like the kind you might find in Haskell or Scala, sort and shuffle have the same type, but they do the opposite of each other. Searching on the basis of functional types is not going to get us very far. But searching on the basis of the behavioral types, whether we’re looking at the Caires-style types or session types, which are slightly weaker than Caires-style types, that’s going to give us a lot more information. In particular, the Caires-style types will give us information about the structure of programs. 

If you look at the way  LADL, for example, generates a logic of monoids, we can detect things that are prime or not composite using that style of type. That notion of primality corresponds to single-threadedness in the Pi calculus or the Rho calculus. You can actually search, you can say, “find me all the programs that are single-threaded.” 

The other way around, more likely, you know that you’ve got a concurrency bug somewhere. “Find me all the programs in my current repo that are multithreaded and have some other property.” That’s going to give you some candidate culprits to go investigate. 

Isaac: You can test the race conditions. 

Greg: That’s a really good example. This gives you a fairly decent discovery mechanism. Now the piece of the puzzle that we haven’t talked about in this is: you can imagine that you can satisfy a type up to some delta. This satisfies most of your type. There’s a notion of residuation that’s at play where you’re saying, it satisfies a kind of implication that involves your type. Now you need to find something that will discharge the hypothesis of the implication. That’s the dependency that you need in order to do that. That gives you the basis for a discovery mechanism where you assemble across this heterogeneous network of smart contracting chains, contracts on the fly, which is where I think we ultimately needed to get to. 

Rolling back from that idea, you can imagine that some contract wants to depend upon another contract that runs on a remote chain. Those contracts are written in heterogeneous languages. But that’s okay because at the level of type—in particular, behavioral-style types—you don’t care what language the thing is written in. All you care about is that it satisfies certain properties with respect to the sequencing of IO. 

Christian: Are you having to compile into Rholang first in order to determine the type? 

Greg: The nice thing is that you let the remote chain semantics provide that. At the end of the day, you’re trusting their consensus algorithm; you’re trusting a whole bunch of things. You have to trust that change attestation that that smart contract satisfies a particular type. Then you go and satisfy yourself that the way it gets to that attestation is a computation that you respect. Does that make sense? 

Christian: You’re querying into other chains that are using other languages and they returned back ones that they have determined to fit that..

Greg: …that spec. That’s exactly right. 

Christian: If you aren’t already providing them with your system, how do they determine? 

Greg: All they have to do is to have the type-checking mechanism between an agreed-upon type of property language and whatever languages they’re expressing their terms. That can be done by a compilation of a common sublanguage or it can be done in a number of other ways.

Your proposal is a particular implementation and I’m trying to be more abstract. I’m trying to say, “if they implemented it in Rust, we don’t care as long as they have a type check,” or “they implemented it in Go, we don’t care as long as they have a type check.” Then it’s up to the community to say whether or not that particular type-checking algorithm is respected. That part of it has been vetted.

Isaac: It almost seems a little implicit because you’re using the data from this chain. You do have to trust all of the aspects that go into it. 

Greg: By the time you’re talking about chain interoperability, especially if you’re talking about smart contract chain interoperability, you’re already there. 

Kudos to Polkadot for recognizing that data is king. If you look at their literature, one of the things that really stands out is how much of their market analysis coincides with our market analysis. Data is king and this will matter enormously if we want to have adoption of the blockchain. 

All of the major players are waiting to see one application other than trading of tokens on top of the blockchain. If the blockchain continues to be optimized just for token moving, it’s setting itself up for a feedback loop that will be very hard to escape. The more it’s optimized for that, the harder it gets to be a data transfer network. Even if you have the greatest interoperability protocol in the world, if the only things that are interoperating are things that have been optimized for moving tokens around, there is no data in that network.

You have to seed such a thing with a data solution that’s scalable enough and fast enough. That’s where RChain landed. You have to have at least one chain that handles data, and also handles sharding, and then extends the sharding solution to an interoperability solution. That feels like a more natural way because then you’re guaranteed at least one of those things is providing an escape path from token to data. 

That’s mostly what I wanted to say. There are other chains that we could talk about. We can talk about Algorand, we can talk about Holochain, but a lot of the criticisms apply. It’s the same kind of critiques. What we’re really demanding (and I think everyone has to demand) is that we have a level of verifiability with respect to the operation of the chain. That is just the new standard. I don’t know how to put it any other way. 

We can think of the last two decades of the internet as exploring all those applications where if you failed it would be okay. If Google Maps doesn’t serve you the right map the first time, it’s okay. If Facebook doesn’t push the cat memes out, that’s certainly okay. If Spotify doesn’t deliver the right song the first time, that’s okay. Nobody is going to die. 

We’re coming up to the end of those markets in some sense. Mostly because we’re running out of time with respect to how to address climate change, but that’s a separate issue. We’re now at a point where we need to consider mission-critical applications and we need to consider the internet-style applications as mission-critical applications. Society is not going backward to a time when it doesn’t have the internet, or at least it doesn’t want to.

We need to think about this next wave of applications in the light of Six Sigma-style stuff. Is your software so good that if it were an air traffic control system, you’d put your daughter on that plane? If that’s your measure, if that’s your bar, then you need to have a level of formal verification. 

The first thing is you’ve got to be able to address the proofs. That’s why I really look for equational reasoning. As soon as the proofs get all fiddly with lots of English language in them, it’s an older style of mathematics. The modern style of mathematics is much more driven in a way that we can automate it with theorem provers and model checkers. 

That’s what you’re driving for and should start to see. That’s what I’ve been trying to argue for in a lot of these talks: look at the Rholang spec. I could go and code this up and in our favorite theorem prover, but I would just want you to see the equational reasoning. Look at it. It’s graspable by ordinary people. That should be the first sign. You start to have these lovely specs that are clean and beautiful. Right on the heels of that, we’ve got theorem provers verifying our work. At that point we can start to talk about mission-critical applications. 

Isaac: That’s always been the draw of the RChain project. You guys are talking about equational reasoning and correct-by-construction and compositionality. That’s what I feel like all of these projects need. 

Greg: I absolutely agree. It’s a funny thing because it is a blind spot. I was talking with someone yesterday, pointing out that people shouldn’t feel bad or embarrassed or even that it’s weird that we don’t have a lot of compositional reasoning in a lot of prominent formulas. It’s been a blind spot in human cognition. Thinking that way takes a particular turn of mind. 

I’ve made this point over and over again, but I think it bears repeating. Quantum mechanics, at least as initially formulated, was not compositional. General relativity—not compositional. That has really big consequences for those theories. The reason I use those examples is that they’re the pinnacle of scientific achievement in terms of accuracy and predictive power. 

In many ways, they’re our gold standard and they’re missing this feature. If you think about them as code bases, they’re missing this feature. It is a new way of thinking to think compositionally, but it’s the only way to get to scale.

Christian: With some of the big chains, each of these systems is going to have its own chain. It’s going to be less common that they go out of their way to form some kind of subchain in another system. When you talk about within RChain having these shards from Ethereum and other places, is there some way that…

Greg: I think I understand your question. There are certain things that we need to get clarity on. Number one, you can have RChain mount Ethereum, so an RChain shard treats Ethereum as a child. You don’t need Ethereum to do anything special. You’ve got an Ethereum contract on one side that’s being backed by an RChain shard on the other. There’s some Ethereum resource that’s backed by some RChain resource. All the Ethereum dApps that use that particular resource, they’re able to do a particular kind of exchange and vice versa. That’s assuming that the resources are effectively ERC-20 tokens. If you want to do RChain and Ether, there’s a little bit more work, but that can also be done in this style without a lot of invasions.

If you want Ethereum to mount RChain, that’s a lot more work. From my point of view, that’s great, because it basically puts RChain in the glue interoperability seat, which is ultimately where you want to be. That kind of landscape is tilted in such a way that all the value rolls out of Ethereum and into RChain. 

There’s no reason why you can’t have multiple Ethereum networks. You just start up a collection of Ethereum nodes and they talk to each other. We already have that. You have Ethereum Mainnet and you have various Ethereum Testnets. It’s only that people recognize one of those networks as the one true Ether. But in point of fact, you could have Red Ether and Blue Ether and Green Ether, and they could all be mounted with respect to each other in such a way that the Red Ether backs the Blue Ether, for example.

Christian: Would a reason why an Ethereum person would come over to the RChain be that RChain starts developing more serious applications first?

Greg: There are two main reasons. One is they start to see that much greater performance in RChain. The second is because there’s greater performance, people are starting to build applications there. CryptoKitties doesn’t bring RChain to its knees. 

Another example is that we’ve measured what it takes to build applications that are serious about data. The RCat backend for RSong and other things enables applications that are data-intensive to get to an MVP in about six to eight weeks with two engineers. That’s about the kind of resource outlay that most projects spend on writing their white paper. But instead of writing a white paper, they could actually produce running code, which demonstrates the point of view. We think that alone could be one of the things that reinvigorates the RChain space.

A lot of developers have left the blockchain space in general because the money has drained out. It’s no longer a gold rush mentality. It’s a different kind of approach, a different kind of space now. If you want to get more dApps involved, you have to show them that there are opportunities for them to be able to write applications that will meet markets that demand things.