## Understanding and Using rindexer for EVM Event Indexing Fetching and displaying historical blockchain event data directly within a frontend application can be slow and inefficient, leading to a poor user experience. The process involves querying the blockchain node repeatedly, which is not optimized for retrieving large sets of past events. A common solution is "indexing," where a dedicated service listens for specific blockchain events, processes them, and stores the relevant data in an easily queryable off-chain database. This lesson introduces `rindexer`, a high-performance, open-source indexing toolset built in Rust. It's designed for compatibility with any EVM (Ethereum Virtual Machine) chain and allows developers to efficiently index smart contract events using a simple YAML configuration file in its `no-code` mode, significantly simplifying the setup for many common use cases. By moving the indexing workload to a dedicated server, `rindexer` provides a fast API (like GraphQL) for your frontend, dramatically improving data retrieval speeds. We will walk through setting up `rindexer` to index events from an example `NftMarketplace` smart contract deployed on a local Anvil network. ## Prerequisites: Tools You'll Need Before starting with `rindexer`, ensure you have the necessary tools installed on your system. ### Docker `rindexer` leverages Docker, particularly Docker Compose, to easily manage associated services like a Postgres database. This is highly recommended, especially if you plan to use Postgres for storage. 1. **Install Docker:** Follow the official instructions at [docker.com/get-started/](https://www.docker.com/get-started/). 2. **Verify Installation:** Open your terminal and run: ```bash docker --version docker ps ``` The first command should output the installed Docker version. The second command lists running containers (it will likely be empty if you just installed Docker). ### rindexer You need the `rindexer` command-line tool itself. 1. **Install rindexer:** Use the official installation script: ```bash curl -L https://rindexer.xyz/install.sh | bash ``` *Note: Always exercise caution when running scripts downloaded from the internet. Review the script content if you have security concerns.* 2. **Verify Installation:** Check if the tool is available in your PATH: ```bash rindexer --help ``` This command should display the help menu listing available `rindexer` commands (`new`, `start`, `add`, etc.). ## Setting Up Your rindexer Project `rindexer` provides a convenient command-line interface (CLI) to scaffold a new project. We'll use the `no-code` mode, which relies on YAML for configuration. 1. **Navigate to Your Workspace:** Open your terminal and go to the directory where you want to create your indexer project. 2. **Create a New Project:** Run the following command: ```bash rindexer new no-code ``` 3. **Follow the CLI Prompts:** * **Project name:** Enter a name (e.g., `marketplaceIndexer`). * **Description:** You can optionally add a description or press Enter to skip. * **Repository:** You can optionally add a repository URL or press Enter to skip. * **What Storages To Enable?:** Use arrow keys and spacebar to select. Choose `both` to enable Postgres and CSV files. Press Enter. * **Postgres Docker Support Out The Box?:** Select `yes`. Press Enter. This process creates a new directory (e.g., `marketplaceIndexer`) containing the necessary configuration files, including `rindexer.yaml` and, because you selected Docker support, a `docker-compose.yml` file to manage the Postgres container. ## Configuring Your Indexer (`rindexer.yaml`) The core of a `no-code` `rindexer` project is the `rindexer.yaml` file. This file tells the indexer *what* to index and *how*. Let's configure it to monitor our `NftMarketplace` contract running on a local Anvil network. 1. **Obtain Contract Information:** You need: * **Contract Address:** The address where your `NftMarketplace` contract is deployed. For local Anvil/Hardhat, this is typically generated during deployment. (Example: `0xe7f1725E7734CE288F8367e1Bb143E90bb3F0512`) * **Contract ABI:** The Application Binary Interface (JSON file) describing the contract's events and functions. This is usually generated during contract compilation (e.g., using `forge build` or `hardhat compile`). Ensure the ABI JSON file is accessible to your indexer project (e.g., copy it into an `abis` subfolder within your `marketplaceIndexer` directory). * *Note:* If using Foundry, you might compile with `forge build`. For complex contracts potentially hitting optimizer limits, a flag like `--via-ir` might be necessary, as mentioned in the video context: `cd foundry && forge build --via-ir`. 2. **Edit `rindexer.yaml`:** Open the `rindexer.yaml` file in your project directory. Modify the default configuration to match your target contract and network. Key sections to update include: ```yaml # Project name (should match what you entered) name: marketplaceIndexer project_type: no-code # Network configuration networks: - name: anvil # Give your network a name (e.g., anvil, localhost) chain_id: 31337 # Chain ID for default Anvil/Hardhat rpc: http://127.0.0.1:8545 # RPC URL of your local node # Storage configuration (reflects CLI choices) storage: postgres: enabled: true # Default connection details usually work with the Docker setup csv: enabled: true path: ./generated_csv # Directory to store CSV files # Contract indexing configuration contracts: - name: NftMarketplace # A descriptive name for your contract indexer details: network: anvil # Must match a name defined in the 'networks' section address: '0xe7f1725E7734CE288F8367e1Bb143E90bb3F0512' # <<< YOUR DEPLOYED CONTRACT ADDRESS HERE # Block to start indexing from. '0' means from genesis. # Crucial for local dev nets to catch all events. start_block: '0' # end_block: ... # Omit this line for continuous 'live' indexing # Path to the contract's ABI file relative to rindexer.yaml abi: ./abis/NftMarketplace.abi.json # <<< PATH TO YOUR ABI FILE HERE # List the specific events you want to index include_events: - ItemListed - ItemCanceled - ItemBought # Add other events from your contract as needed ``` 3. **Save the File:** Ensure you save your changes to `rindexer.yaml`. ## Running the Indexer With the configuration complete, you can now start the indexer service. 1. **Navigate to Project Directory:** Make sure your terminal is inside the indexer project directory (e.g., `marketplaceIndexer`). 2. **Start the Indexer:** Run the command: ```bash rindexer start indexer ``` **What Happens Now?** * `rindexer` reads `rindexer.yaml`. * If Docker support is enabled and Postgres is selected, it will likely use `docker-compose up` in the background to start the Postgres container (if not already running). *Note: The very first time you run this, you might see a connection error if the Postgres container hasn't fully initialized. Simply wait a few seconds and run `rindexer start indexer` again.* * It connects to the specified network RPC URL (`http://127.0.0.1:8545`). * It reads the specified ABI (`./abis/NftMarketplace.abi.json`) to understand event structures. * It connects to the configured storage (Postgres database and/or sets up the CSV directory). * It starts scanning the blockchain from the `start_block` ('0' in our case). * It processes blocks, looking for events specified in `include_events` (`ItemListed`, `ItemCanceled`, `ItemBought`) emitted by the target contract address. * For each event found, it decodes the data using the ABI and saves it to the configured storage (Postgres tables and/or CSV files). * Once it reaches the current block, it switches to `LIVE` mode, continuously listening for new blocks and indexing relevant events in real-time. You will see log output in your terminal indicating the progress, blocks being processed, and events found. ## Verifying the Indexed Data How do you know it's working? 1. **Check Console Logs:** The `rindexer start indexer` command outputs detailed logs showing block numbers processed, any events indexed in those blocks, and status updates (e.g., `HISTORICAL SYNC`, `LIVE`). 2. **Inspect CSV Files:** If you enabled CSV storage, navigate to the specified `path` (e.g., `./generated_csv`). You should find CSV files named after the contract and event (e.g., `nftmarketplace-itemlisted.csv`, `nftmarketplace-itembought.csv`). Open these files to see the indexed event data in a tabular format. This is a quick way to visually confirm data capture. 3. **Query Postgres Database:** If you enabled Postgres, the data is stored in tables within the database managed by Docker. You would typically use a database client tool (like `psql`, DBeaver, pgAdmin) to connect to the database (using connection details potentially found in `docker-compose.yml` or `rindexer` defaults) and query the tables corresponding to your events. `rindexer` automatically creates tables based on your contract events. ## Why Use rindexer? The Benefits By running `rindexer`, you have created an off-chain, indexed representation of your smart contract's event history. The primary benefit is performance: * **Fast Frontend Queries:** Your frontend application no longer needs to make slow, cumbersome calls directly to the blockchain node to fetch historical data. * **Dedicated API:** `rindexer` typically exposes an efficient API (often GraphQL, though configuration details may vary) that your frontend can query. This API accesses the optimized Postgres database or other storage, returning data much faster. * **Simplified Frontend Logic:** Data retrieval logic on the client-side becomes simpler, focusing on querying the indexer's API rather than handling complex blockchain interactions for historical state. * **Scalability:** Off-loading indexing to a dedicated service improves the scalability of your application. `rindexer` provides a powerful yet accessible way (`no-code` mode) to implement this crucial pattern in web3 development, significantly enhancing application performance and user experience when dealing with smart contract events. For more complex transformations or logic during indexing, `rindexer` also offers a `rust` mode (`rindexer new rust`) allowing developers to write custom indexing logic using the Rust programming language.