The Evolution of Blockchain Data Indexing Technology: From Node to Full Chain Database

1. Introduction

Since the first batch of dApps emerged in 2017, blockchain applications have flourished in various forms. With artificial intelligence and Web3 becoming the focal points in 2024, the importance of data to AI systems is self-evident. This article will delve into the development of blockchain data accessibility and compare the similarities and differences between traditional data indexing protocols and emerging blockchain data service protocols in terms of data services and product architecture.

2. The Evolution of Data Indexing: From Blockchain Nodes to Full-Chain Database

2.1 Data Source: Blockchain Node

Blockchain nodes serve as the foundation of a decentralized network, bearing the heavy responsibility of recording, storing, and disseminating on-chain transaction data. However, ordinary users find it difficult to establish and maintain nodes, and thus typically rely on third-party services. RPC node providers have emerged to offer data access services through RPC endpoints. Nevertheless, RPC endpoints still face inefficiencies in complex data queries.

2.2 Data Parsing: From Raw Data to Usable Data

The raw data provided by Blockchain Nodes is often encrypted and encoded, making it quite challenging for ordinary users and developers to use this data directly. The importance of the data parsing process is thus highlighted, as it transforms complex raw data into a more understandable and operable format, which is a key link in the entire data indexing process.

2.3 The Evolution of Data Indexers

As the volume of Blockchain data surges, the demand for data indexers is becoming increasingly urgent. Indexers organize on-chain data and store it in databases, providing data access services using SQL-like query languages (such as GraphQL). Different types of indexers include full node indexers, lightweight indexers, dedicated indexers, and aggregated indexers, each optimized for different scenarios.

Compared to traditional RPC endpoints, indexers have significantly improved data indexing and query efficiency. They support complex queries, data filtering, and analysis, with some able to aggregate multi-chain data, greatly simplifying the development process of multi-chain applications.

2.4 Full Chain Database: Aligning to Stream Priority

As application demands grow increasingly complex, standardized APIs struggle to meet diverse query needs. Blockchain data service providers are beginning to adopt a "stream-first" approach, building blockchain data streams to enable real-time data ingestion, processing, and analysis. This approach allows organizations to respond to data almost instantaneously, thereby enhancing decision-making efficiency.

3. AI + Database: In-depth comparison of The Graph, Chainbase, and Space and Time

3.1 The Graph

The Graph provides multi-chain data indexing and querying services through a decentralized network of nodes. Its core product model includes a data query execution market and a data indexing cache market. The network consists of four key roles: indexers, curators, delegators, and developers, who ensure the system operates through economic incentives.

Recently, The Graph ecosystem has introduced several AI-driven tools, such as AutoAgora, Allocation Optimizer, and AgentC, to optimize index pricing, resource allocation, and user query experience.

3.2 Chainbase

Chainbase, as a full-chain data network, integrates all blockchain data. Its features include a real-time data lake, a dual-chain architecture based on Eigenlayer AVS, an innovative "manuscripts" data format standard, and a cryptographic world model Theia created in conjunction with AI model technology.

Theia is based on NVIDIA's DORA model, which combines on-chain and off-chain data to deeply analyze encryption patterns and provide users with intelligent data services.

3.3 Space and Time

Space and Time (SxT) is committed to creating a verifiable computing layer, achieving zero-knowledge proofs on decentralized data warehouses through Proof of SQL technology. This innovative approach ensures the integrity and accuracy of SQL query results, providing trustworthy data processing services for smart contracts, large language models, and enterprises.

SxT has also collaborated with Microsoft's AI Innovation Lab to develop generative AI tools that simplify the process for users to process blockchain data through natural language.

Conclusion and Outlook

Blockchain data indexing technology has evolved from node data sources to AI-enabled full-chain data services. In the future, with the development of new technologies such as AI and zero-knowledge proofs, blockchain data services will become further intelligent and secure, continuing to support industry innovation as infrastructure.