What is Gavin Wood's officially announced next-generation Polkadot technology vision 'JAM”?

On April 19, at the Token 2049 conference in Dubai, Gavin Wood announced a bold vision for the next generation of Polkadot technology. Like the other groundbreaking firsts Polkadot has brought to the market, this new vision will revolutionize the future of Web3. It will provide the speed, scale, complete decentralization, and ease of use that Web3 needs to drive deep innovation in Web3 and the entire technology field.

At the heart of this vision is JAM, a new version of the Polkadot chain that will push Polkadot’s capabilities beyond the current boundaries of Web3 while making it possible to deploy a wide range of technologies on Polkadot. With JAM, breakthrough scalability currently only seen through rollups is brought to the consensus layer.

When developed, JAM will be a distributed computer that can run almost any type of task that can be represented as a service. JAM pushes Polkadot into the world of synchronous composability, which will help reduce fragmentation and consolidate activity so applications on Polkadot can better leverage the network across the ecosystem. This will open up new possibilities for deep innovation and provide developers with a powerful environment to create in ways never possible before.

JAM is currently in the research and development stage. Currently, the Polkadot community has a proposal up for vote (Referendum 682 https://polkadot.polkassembly.io/referenda/682) to confirm this new direction and authorize the Technical Fellowship to approve JAM.

In order to support the development of JAM and ensure that it is built in the spirit of true decentralization, Gavin jointly announced the establishment of the JAM bonus with the Web3 Foundation in his speech, with a total of 1,000 10,000 DOT will be used to stimulate the development of other implementations of JAM.

At the same time as the speech, Gavin also released a technical gray paper. If you want to dive deeper into the vision and technical details of the project, the paper can be found on the new JAM Graypaper website: https://graypaper.com/.

Together, Gavin and Polkadot lead the creation of innovative technologies designed to realize the vision of creating a free and open web. JAM is the next chapter in the evolving Polkadot story.

The following is the latest introduction to JAM from the Polkadot Wiki, written by Gavin Wood.

The full name of JAM is Join-Accumulate Machine, which is a new design planned to replace the existing relay chain. The name JAM is derived from CoreJAM, which stands for Collect Refine Join Accumulate and describes the computational model embodied by the machine, originally described in an RFC by Gavin Wood. However, in the actual chain, only Join and Accumulate are performed, while the Collect and Refine processes occur off-chain.

Unlike the current iterative approach, JAM will be introduced as a comprehensive single upgrade. A few reasons for doing this include:

• Uniform upgrades can precisely constrain post-upgrade operations, which is difficult to do in an iterative approach.
• It reduces the small upgrades and major changes that often occur regularly over weeks or months.

While this transition will require significant breakthrough changes, we will work to reduce its impact to a manageable level. It is better to consolidate multiple smaller breaking changes into a single transition, which introduces a new blockchain concept and integrates various existing ideas.

A Rollup chain

JAM will be a domain-specific chain used to deal with problems in a specific domain. In this case, it's a rollup, which the Ethereum community calls optimistic roll-up. JAM's rollup is very limited in terms of security. This is what Polkadot has been doing for the past five years, and it has become a highly domain-specific rollup chain. JAM essentially makes it more versatile with fewer preset preferences.

The JAM chain takes the output of the rollup, and more generally the bits of computation done elsewhere, integrating the output into a shared state, similar to the functionality of the Polkadot relay chain.

The job of the JAM chain is to provide the necessary equipment to ensure that the output correctly reflects the input after being transformed.

Similarities with smart contract chains

JAM has several similarities with smart contract chains:

• The JAM chain itself directly executes permissionless code .
• The state of the JAM chain is organized into different packages.
• In addition to the encapsulation of status, it also includes the encapsulation of code and balance.

The encapsulation of these states is called a service. Therefore, the status of JAM is divided into services. Creating new services is permissionless, similar to deploying smart contracts on a smart contract chain. Therefore, adding new services to a JAM chain does not require any authoritative approval or compliance with governance mechanisms, unlike Substrate-based chains, which require governance approval for the addition of new pallets. Services include code, balances, and certain state components, similar to structures commonly seen on smart contract chains.

Service entry point

The code of the JAM service is divided into three different entry points:

• Refine is the function that performs most stateless calculations. It defines transformations for rollups of specific services.
• Accumulate function takes the output and folds it into the overall state of the service
• OnTransfer handles information from other services.

The work package is the input to the service. A work package can contain many work items. Each work item is associated with a service, and it reflects actual input to that service. For parachain services, this is where transactions and all blockchain inputs come in.

JAM's safety device consists of two-stage processing, where the Refine function accepts work items as input and produces the results of the work as output, and then enters the Accumulate function (first Refine, then Accumulate). Work items are refined into work results. Therefore, a work package containing many work items is refined into work reports. The work report is the result corresponding to multiple work items. A work package can be assigned to use a core for a specific period of time (usually 6 seconds).

JAM is transactionless

JAM is distinguished from the smart contract chain through transactionless operations. There are no transactions within JAM; all actions are permissionless and initially go through the Refine phase. At this stage, the service pre-refines the input data and converts it into a work report containing the results of the work. The results of these efforts are then transferred on-chain.

Despite no transaction, JAM still accepts external information in a specific format. There are five types of external information:

1. Guarantees

2. Assurances

3. Judgments

4. Preimages

5. Tickets

The first three types are part of the JAM chain security framework. "Guarantees" and "Assurances" involve validators collectively proving that a work result accurately reflects the result of its corresponding work item when transformed through the service's "Refine" function.

Judgment occurs when the integrity of a work result is called into question, when a large number of validators attest to its validity, or lack thereof. In this case, invalid work items may have been incorporated into the service's state and may need to be rolled back. Judgment must be made within one hour of submitting the work report to the chain, during which the block is temporarily suspended.

The original is a feature provided by the JAM chain for the Refine function. Although the Refine function is typically stateless, it can perform one stateful operation: looking up a hashed preimage. This feature is the only aspect of the Refine function that has a default preference.

The tickets enter the block production mechanism as anonymous entries. They are not direct requirements for block production; instead, the system operates two epochs in advance. This mechanism is part of the SAFROL algorithm, a refined version of the original SASSAFRAS algorithm.

Refine Function

In JAM, the Refine processing stage can accept up to 15 MB of data per time period, each time period lasting 6 seconds. However, the data produced by Refine is up to 90 kB, requiring extensive data compression due to the distributed nature of the availability system. For example, in the context of parachains, 15 MB of data represents a Proof of Validity (PoV), while 90 kB of data corresponds to a candidate receipt.

Refine can use up to 6 seconds of PVM Gas, equivalent to the full block period of the relay chain. This extended execution time, compared to PVF's current two-second limit, is achieved through safety metering and other optimizations.

Refine can also perform original image search. If a hash and its associated preimage are believed to be available on the JAM chain, the preimage can be requested by providing the hash. This capability enables efficient storage and retrieval of code, for example by storing parachain code on the JAM chain and referencing its hash in the work package.

Refine is the main processing power and handles most of the tasks that are stateless operations.

Accumulate Function

The Accumulate function is responsible for integrating the output generated by the Refine function into the chain state. Accumulate can accept multiple outputs from Refine, all originating from the same service. Refine and Accumulate both serve as entry points from a specific service code block.

Accumulate's execution time per output is much shorter than Refine's, typically only 10 milliseconds at most. However, the duration depends on the number of work items in the work package. If a work package contains multiple items, the available time is divided among them.

Unlike Refine, Accumulate is stateful and can access the state of the JAM chain. It can read storage from any service, write to its key-value store, transfer funds, and include memos when funds are transferred. Additionally, Accumulate can create new services, upgrade their code, request the availability of preimages, and more.

In addition, Refine can call sub-instances of PVM. This allows the creation of sub-instances or virtual machines where code and data can be deployed, memory and stack configurations customized, and computations performed within a flexible framework.

onTransfer function

The onTransfer function in the JAM system is also stateful, allowing it to modify the state of the service. It has the ability to check the status of other services and modify its own status. This feature facilitates communication between services, albeit in an asynchronous manner.

Unlike many smart contract platforms where interactions occur synchronously, in JAM, interactions between encapsulated components (such as smart contracts or services in this case) occur asynchronously. Messages and tokens are sent together, and at some point within the same six-second execution cycle, they are processed by the receiving service. There is no immediate return path; if a return path is required, the sending service must initiate another transfer or modify its state in a way that the receiving service can later interpret.

Accumulate and onTransfer are both designed to be executed in parallel, allowing Accumulate and transfer of different services to occur simultaneously. This design opens the possibility for future enhancements, such as allocating gas input beyond the current 10 milliseconds. In theory, a secondary core could be used to perform certain Accumulates, giving them more gas to exploit.

Generalization of JAM chains

As stated in the original Polkadot white paper, Polkadot is primarily customized for specific service profiles: providing parachains. To implement this service, Polkadot developed two important sub-components:

• Distributed Data Availability System
• A system that provides auditing and assurance for computation (i.e. with strong security guarantees Optimistic rollup system)

Compared to Polkadot, JAM has fewer preset preferences and provides a higher level of abstraction and generalization. This makes it easier to leverage underlying components based on personal preference.

JAM operates in a permissionless manner, similar to a smart contract chain, allowing individuals to upload and execute intended code. Additionally, it hosts data, enables preimage lookups, and manages state, similar to a key-value system. Since JAM lacks a mechanism to directly accept transactions, JAM's genesis block contains a service that facilitates the creation of new services.

Services within JAM have no predefined limits on the amount of code, data, or state. Their capabilities are determined by cryptoeconomic factors; the more DOT tokens deposited, the greater the capacity for data and state. For example, the parachain service on JAM consolidates all Polkadot 1.1 functionality into a single service, and other services can also take advantage of Polkadot's distributed availability system, as well as audit and assurance systems for computation.

Polkadot Virtual Machine (PVM)

The design of PVM is based on the RISC-V Instruction Set Architecture (ISA), which is known for its simplicity and versatility. The RISC-V ISA offers several advantages:

1. Easy translation to common hardware formats such as x86, x64, and ARM.

2. Get good support from tools like LLVM.

PVM itself embodies simplicity and security, has the ability to be sandboxed and provides various execution guarantees. It is deterministic, consensus-sensitive, and easy to measure. Compared to other virtual machines, PVM lacks complexity and excessive preset preferences.

WASM, while optimized for web use cases, also presents stack management challenges, especially when it comes to handling continuity. RISC-V solves this problem by placing the stack in memory, thus naturally facilitating sequential processing without adding complexity.

Additionally, PVM demonstrates superior execution speed when running on legacy hardware, especially on X64 and ARM, providing benefits such as free metering, which compares favorably to WASM.

Support for RISC-V continuity will establish a new standard for scalable coding across multi-core platforms like JAM. Asynchronous parallel architectures are increasingly important for the scalability of hardware and software platforms, a trend expected to extend to blockchains and consensus algorithms.

SAFROLE

SAFROLE is a block production algorithm that simplifies SASSAFRAS. It excludes some components that may be useful for parachains. So parachains may stick to SASSAFRAS instead of SAFROLE. SAFROLE will be as simple as possible:

• Ensure that there are as few preset preferences as possible to maximize potential future use cases.
• Follow in the footsteps of the Ethereum Yellow Paper and really try to get as many implementations out there to try and spread the expertise.

Understanding how Polkadot 1.0 works end-to-end is challenging. With JAM, someone who can read and understand the yellow paper should be able to read and understand how JAM works very quickly. So simplicity is crucial.

SAFROLE is a block production algorithm based on SNARK. It uses SNARKs because of their anonymizing properties. And it provides almost completely fork-free constant-time block production. The few instances where a fork can occur basically only happen when there is a network split or someone acts intentionally maliciously. The great value in anonymity is not to keep the identity of the validators secret; in fact, when they produce a block, they reveal their identity anyway, and this is done to ensure the security of the block production mechanism itself, basically To avoid spam transaction attacks.

Network

JAM’s network uses the QUIC protocol. This allows direct peer-to-peer connections between large numbers of validators. As a result, the 1000+ validators on Polkadot can maintain persistent connections to each other without having to worry about possible socket issues. Since JAM does not handle transactions, there is basically no need for gossip. In situations where distribution is required that is not point-to-point or within a very small subset of validators, grid diffusion is used, where the validators are arranged into a grid, packets are sent in rows, and then each node sends it to all of its columns member.

Pipeline for efficient block processing

In state-based blockchains like Ethereum, the header of the block usually contains the post-state root, which summarizes all block calculations the final state. Therefore, the block header cannot be sent until all calculations are completed. However, some calculations can be performed before sending the block header, as their results determine the validity of the block.

However, JAM takes a different approach, placing the pre-state root in the block header instead of the post-state root. This means that the state shown in the header is delayed by one block. As a result, lightweight computations accounting for approximately 5% of the block workload or execution time can be performed and blocks can be distributed immediately. The remaining 95% of block calculations, mainly Accumulate tasks, can be completed later. This allows the next block to be started before the current block is executed.

This approach allows for more efficient use of the time between blocks. In a traditional setup, such as Polkadot's six-second block time, the post-state root must be provided and can only be computed part of the time. However, through the pipeline in JAM, the entire block time can be effectively utilized for calculations, maximizing efficiency.

While using the entire block time for calculations may not be ideal as it may result in permanent catch-up and delayed block imports, JAM's approach can significantly extend calculation times compared to traditional setups. This means that effective block calculation times of around three to three and a half seconds can be achieved, which is a huge improvement over current setups.

Architectural Difference: JAM vs. Relay Chain

One architectural difference between JAM and relay chain is the degree of fixed functionality. While the relay chain fixes some elements, such as the language used to define the protocol (WASM), JAM goes further in this regard. For example, it dictates the types used to encode block headers and hashing schemes, making it difficult to change these aspects.

However, JAM retains flexibility through its service model comparable to that achieved by the WebAssembly meta-protocol in the relay chain. In this model, the responsibility for upgradeability is shifted to the service, freeing the chain itself from the burden of upgrades. Three main reasons supporting this decision include:

1. Prioritize simplicity. Maintaining non-upgradeable chains significantly reduces complexity.
2. Relay chains tend to introduce complex functionality without removing old functionality, which complicates things. Since upgrades are easy to implement, there is little incentive to simplify the Substrate SDK. Therefore, replicating Polkadot becomes impractical.
3. JAM's fixed parameters provide optimization potential. By having a clear understanding of the specific tasks that the JAM chain must perform and being able to set fixed parameters, optimization in areas such as network topology and timing becomes feasible. This contrasts with the highly upgradeable nature of the relay chain, where these optimizations are more complex due to the frequent changes likely to be made with each upgrade.

Despite these differences, JAM maintains flexibility in application-level functionality such as core time sales, staking, and governance, all managed within the service. Additionally, JAM introduces a new concept by associating token balances with services, providing opportunities for economic model adjustments that are not easily achieved in purely upgradeable chains such as relay chains.

JAM Toaster

To ensure that JAM lives up to its original expectations, an ongoing effort includes establishing a comprehensive testing environment for the JAM chain. Rather than running a small-scale test network on unreliable hardware to manage cloud computing costs, this initiative involves a significant investment.

JAM Toaster is introduced here, which is essentially a testing platform for large-scale experiments and performance evaluation. This addresses previous challenges encountered during the development of the Polkadot Relay Chain, when we learned that operating the network’s emergent effects and dynamics at such a scale proved difficult. Previous attempts were limited to a few dozen nodes on the test network and the Kusama network, which lacked comprehensive monitoring capabilities due to lack of access to validating nodes. In contrast, small-scale test networks fail to accurately simulate the network dynamics of large-scale deployments.

JAM Toaster aims to bridge this gap by providing an in-depth look at the entire JAM network, including 1023 nodes. The platform facilitates the study of network behavior and performance characteristics, providing developers with valuable insights into the expected performance of parachains.

JAM and Substrate

Benchmarking vs Metering

In JAM, benchmarking or performance testing can be optional. While benchmarking may still be required in some cases, JAM's metering system generally eliminates the need for frequent benchmarking. JAM runs on a metering system that allows users to evaluate computational workload upon completion. Additionally, there is a theoretical mechanism to control computation when blocks are constructed.

However, in some cases, benchmarking is still required. For example, when metering is too conservative for certain use cases, benchmarking may be needed to improve performance. Additionally, benchmarks are useful for tasks that require extended execution times to ensure they don't run for too long.

XCMP

Next up is Cross-Chain Messaging (XCMP), and JAM requires full XCMP support. This is because in a relay chain, if all parachains transmit all data all the time, more data can be passed through candidate receipts. JAM strictly adheres to the rules, even for parachain services, including restrictions on data transfer between the “Refine” and “Accumulate” phases. Currently, using Horizontal Relay Chain Messaging (HRMP), all messages traverse the relay chain, limiting the data payload to 4 kB or less, which may not be practical. As a result, XCMP only relays message headers through the chain while the actual message data is transmitted off-chain, a necessary and long overdue improvement.

Accords

Accords essentially encapsulate state and logic, similar to smart contracts, with multiple instances of each parachain. They facilitate the exchange of messages between instances and enable synchronous interaction with parachains. The protocol is useful in scenarios where there is a lack of trust between parachains, such as token transfers. Unlike existing approaches involving reserve intermediaries, Accords simplifies direct token transfers between parachains, eliminating the need for trust-compromising intermediaries. Additionally, Accords can act as an XCM forwarding mechanism to ensure message integrity even when relayed through third-party intermediaries, thereby eliminating the need for explicit origin tagging.

Improving Efficiency

Finally, JAM takes a broader, less pre-preferred approach to leveraging the underlying consensus mechanism, helping to implement more innovative solutions. For example, for complex tasks like zero-knowledge proofs, distributed availability becomes more practical and efficient. Additionally, JAM supports a mixed resource consumption model, where work packages can contain both compute-intensive tasks and data-intensive operations. By pairing compute-intensive services with services that require high data availability, JAM optimizes the utilization of validator resources, thereby reducing costs. This flexible approach makes combining workloads such as distributed availability and SNARK verification with parachains less expensive while increasing efficiency.

JAM Enhancements and Compatibility

JAM was designed to prioritize compatibility with existing Polkadot 1 parachains. While it maintains compatibility with the Polkadot SDK, the Polkadot Validator Function (PVF) has been relocated. It will target the Polkadot Virtual Machine (PVM), not WebAssembly. This transition is facilitated by the fact that PVM is a slight modification of RISC-V, which has been formally identified as a target of LLVM. Therefore, PVM may become an official LLVM target before JAM is deployed.

In addition to serving as a host for parachains, JAM introduces significant enhancements. It offers the potential to simplify benchmarking efforts and alleviate future benchmarking needs. In addition, JAM introduces the concept of protocols, multi-instance, and multi-shard smart contracts to manage and execute specific interaction protocols between parachains. Additionally, comprehensive Cross-Chain Messaging (XCMP) support is crucial, making it possible to remove restrictions on information transfer between parachains, which is typically facilitated through Cross-Chain Messaging (XCM).

Regarding Agile Coretime, JAM maintains compatibility with existing settings. However, it introduces the ability to locate core times not just on parachains, but on any group of work packages. This flexibility enhances the versatility and efficiency of resource allocation within the JAM ecosystem.

The above is the detailed content of What is Gavin Wood's officially announced next-generation Polkadot technology vision 'JAM”?. For more information, please follow other related articles on the PHP Chinese website!