OnFinality’s New Trace API - Helping Colorful Notion get trace data from over 40 Substrate chains

OnFinality is a blockchain infrastructure platform that saves web3 builders time and makes their lives easier. We deliver easy-to-use, reliable and scalable API endpoints for the biggest blockchain networks and empower developers to automatically test, deploy, scale and monitor their own blockchain nodes in minutes.

In this Case Study, we hone in on how our new Trace API feature helped Colorful Notion (the developers of Polkaholic.io and substrate-etl) get storage changes, critical to XCM Indexing and deep indexing of Polkadot and Kusama parachains, published in BigQuery here.

“Storage changes exposed by Onfinality’s Trace API (state_traceBlock) is critical to indexing the entire Substrate ecosystem effectively, from linking XCM messages to monitoring how underlying storage maps changes.  Colorful Notion’s Polkaholic indexer uses Onfinality’s RPC public endpoints across the ecosystem, but until this Trace API, the “unsafe” state_traceBlock were not available, requiring us to run several dozen full nodes. Working together with OnFinality, Colorful Notion can reduce the need to run expensive full nodes while increasing coverage.”

— Sourabh Niyogi, Cofounder/ CEO, Colorful Notion, the developers of Polkaholic.io

Summary

OnFinality recently added a new Trace API feature to empower developers in the Substrate ecosystem with greater control over their applications. During our beta, we’ve been working closely with key partner developers to test and optimise the new Trace API endpoints based on real-world usage.

The tests were done with Colorful Notion, the developers of Polkaholic.io, one of OnFinality’s long-time customers.

“Colorful Notion is a top-notch team that builds with a collaborative development approach and are eager to try out innovative solutions. They have always been highly responsive, which was crucial to the success of the tests.”

— Daniel Vulinovich, Product Manager at OnFinality

What is Polkaholic?

Polkaholic.io is a leading multi-chain block explorer serving users and developers of Polkadot / Substrate chains with data not found in other indexers. It provides key block, extrinsic, event, account, and XCM data through its multichain UI, over its API, and via public datasets in Google BigQuery.

The Challenges Colorful Notion Faced

Colorful Notion currently runs 30 nodes in a Kubernetes cluster to index XCM and traces. Running full nodes is expensive and time-consuming to maintain. As XCM increases its footprint in new chains, especially for new parachain, it is challenging to justify maintaining new nodes.  

Colorful Notion asked OnFinality about systemic support for “unsafe” calls in Winter 2023, and OnFinality saw that it could support Colorful Notion’s indexing efforts across the Substrate ecosystem given its long-standing history in maintaining full archive nodes with enterprise grade support across multiple cloud systems and across the globe.  

In Spring 2023, the OnFinality team and Colorful Notion worked together to test “state_traceBlock” which at first acted as a backup to Colorful Notion but is now the primary source for a dozen nodes.

The Testing Process

We worked closely with Colorful Notion to test out the feature across 15 different networks over a period of 2 months, during which we managed to determine the load capacity and best practices for using the Trace API.

How OnFinality’s New Trace API Helped

By using the Trace API and migrating their 2022 XCM Transfer analysis to the BigQuery public datasets, Colorful Notion was able to get the data they needed in real time for a fraction of the cost of running a node. This allowed them to achieve their goals in a more cost-effective and efficient manner.

Key Findings

Together, we identified two best practices for optimal utility of the tracing feature.

1. It was important to understand the load on OnFinality's servers in order to run a tracing API on Polkadot and its parachains.
2. Defining the appropriate weight was crucial to achieving cost and performance efficiency for Colorful Notion.

The Results

OnFinality's Trace API feature reduced Colorful Notion’s need to run full nodes just to get a few “unsafe” calls.  By leveraging OnFinality's Trace API, they were able to achieve the same indexing at a reduced fraction of the cost for new chains and adding.

Other Substrate teams that need to reason about storage changes, such as Decentralised Exchanges (DEXes) and cross-chain platforms, can also benefit from this feature.

“OnFinality’s Trace API enables Colorful Notion to build more comprehensive indexing as OnFinality approaches 100% coverage of the Substrate ecosystem. We enjoy working with Onfinality’s stellar team and look forward to collaborating on similar problems in a Substrate+EVM multichain future.”

— Sourabh Niyogi, Cofounder/ CEO, Colorful Notion, the developers of Polkaholic.io

How you can access OnFinality’s Trace API

Reach out to us at sales@onfinality.io to try out the new OnFinality Trace API now!

Official Release

OnFinality’s Trace API is slated to be publicly available in June, follow us on Twitter to be notified of the latest chain and feature updates!

Read the beta launch announcement here.

Resources

https://documentation.onfinality.io/support/trace-api

Build Smarter with OnFinality 😎

About OnFinality

OnFinality is a blockchain infrastructure platform that saves web3 builders time and makes their lives easier. OnFinality delivers scalable API endpoints for the biggest blockchain networks and empowers developers to automatically test, deploy, scale and monitor their own blockchain nodes in minutes. To date, OnFinality has served over hundreds of billions of RPC requests across 80 networks including Avalanche, BNB Chain, Cosmos, Polkadot, Ethereum, and Polygon, and is continuously expanding these mission-critical services so developers can build the decentralised future, faster!

App | Website | Twitter | Telegram | LinkedIn | YouTube