> ## Documentation Index > Fetch the complete documentation index at: https://covalenthq.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Run BRP Node As discussed in the setup section, there are two ways to setup the node and so accordingly these instructions are below: 1. **Run with Docker Compose** - (Recommended method - Beginner) 2. **Build & Run from Source** - (Optional method - Advanced) ### 1. Steps to Run Node with Docker: Note: The current **BRP-Refiner** stable version is **0.6.1**. For the latest updates or additional details, please check the announcements or discussions in **Discord**. Clone the [covalenthq/refiner](https://github.com/covalenthq/refiner/) repo. ```shell theme={null} git clone https://github.com/covalenthq/refiner cd refiner cat docker-compose-mbeam.yml ``` ```shell theme={null} $ cat .envrc export IPFS_PINNER_URL="http://ewm-das:5080/api/v1" export EVM_SERVER_URL="http://evm-server:3002" [[ -f .envrc.local ]] && source_env .envrc.local ``` ```shell theme={null} $ touch .envrc.local export BLOCK_RESULT_OPERATOR_PRIVATE_KEY="<>" export NODE_ETHEREUM_MAINNET="<>" export IPFS_PINNER_URL="http://ewm-das:5080/api/v1" export EVM_SERVER_URL="http://evm-server:3002" export W3_AGENT_KEY="<>" export W3_DELEGATION_FILE="<>" ``` **Note:** `.envrc.local` overrides any env vars set with `.envrc` on calling `direnv allow` . **Note:** When passing the private key into the env vars as above please remove the `0x` prefix so the private key env var has exactly 64 characters. * `BLOCK_RESULT_OPERATOR_PRIVATE_KEY:` Your personal Block Result Producer (BRP) operator private key * `AGENT_KEY & PROOF.OUT File:` Your personal web3.storage Agent key & Delegation file use by the `ipfs-pinner` service * `IPFS_PINNER_URL:` Service (`ipfs-pinner`) used by rudder to access IPFS assets like Block Specimens (service is automatically invoked and run with the in repo docker compose files eg: `docker-compose-mbeam.yml`) * `EVM_SERVER_URL:` Service (`evm-server`) used by rudder for stateless execution of Block Specimens into indexable (queryable) Block Results #### Start Services ```shell theme={null} $ direnv allow . #make sure you see these being loaded direnv: loading ~/refiner/.envrc direnv: loading ~/refiner/.envrc.local direnv: export +BLOCK_RESULT_OPERATOR_PRIVATE_KEY +EVM_SERVER_URL +IPFS_PINNER_URL +NODE_ETHEREUM_MAINNET +W3_AGENT_KEY +W3_DELEGATION_FILE ``` Copy over the delegation proof file to `~/.ipfs` repo. You should have gotten this from Covalent in addition to the `W3_AGENT_KEY` value. ```shell theme={null} mv path_to_delegation_file ~/.ipfs/proof.out ``` ```shell theme={null} $ docker compose -f "docker-compose-mbeam.yml" up -d --remove-orphans [+] Running 4/4 ⠿ Container refiner Started 3.2s ⠿ Container autoheal Started 0.5s ⠿ Container das-pinner Started 1.9s ⠿ Container evm-server Started 1.8s ``` NOTE: On a system where an `das-pinner` instance is already running, check the instruction in the [Appendix](https://www.covalenthq.com/docs/covalent-network/operator-onboarding-refiner/#appendix) to run `refiner` docker alongside. Monitor the logs for Block Result submissions. ```shell theme={null} $ docker compose -f "docker-compose-mbeam.yml" logs -f –tail 2 refiner | [info] curr_block: 4591264 and latest_block_num:4591263 das-pinner | 2023/06/22 13:45:49 Received /health request: source= 127.0.0.1:54420 status= OK refiner | [info] curr_block: 4591264 and latest_block_num:4591263 das-pinner | 2023/06/22 13:46:00 Received /health request: source= 127.0.0.1:54430 status= OK refiner | [info] curr_block: 4591264 and latest_block_num:4591264 refiner | [info] listening for events at 4591264 refiner | [info] found 0 bsps to process ``` **Check step 7 in Run BRP** from source section below for a successful Refiner stack run log output with performance metrics. #### Deployment As Service Unit: Here you can find an example of a `systemd` service unit file that can be used to auto-start/restart of the docker-compose service for Refiner. ```shell theme={null} [Unit] Description=Refiner docker compose PartOf=docker.service After=docker.service export IPFS_PINNER_URL="http://ewm-das:5080/api/v1" export EVM_SERVER_URL="http://evm-server:3002" [Service] User=blockchain Group=blockchain Environment=HOME=/home/blockchain/tmp Environment="BLOCK_RESULT_OPERATOR_PRIVATE_KEY=<>" Environment="NODE_ETHEREUM_MAINNET=<>" Environment="IPFS_PINNER_URL=http://ewm-das:5080/api/v1" #service in docker Environment="EVM_SERVER_URL=http://evm-server:3002" #service in docker Environment="W3_AGENT_KEY=<>" Environment="W3_DELEGATION_FILE=<>" Type=simple ExecStart=docker compose -f "/home/blockchain/tmp/docker-compose-mbeam.yml" up --remove-orphans Restart=always TimeoutStopSec=infinity [Install] WantedBy=multi-user.target ``` After adding the env vars in their respective fields in the service unit file, enable the service and start it. ```shell theme={null} sudo systemctl enable refiner-compose.service sudo systemctl start refiner-compose.service ``` **Note 1:** When passing the private key into the env vars as above please remove the 0x prefix so the private key env var has exactly 64 characters. **Note 2:** In order to run docker compose as a non-root user for the above shown service unit you need to create a docker group (if it doesn't exist) and add the user (say) "blockchain" to the docker group ```shell theme={null} sudo groupadd docker sudo usermod -aG docker blockchain sudo su - blockchain docker run hello-world ``` **Hurray!** You have Successfully implemented the BRP Node using Docker Compose #### 2. Steps to Run Node with Source: Here, to run the node from source we need to manually run all the three services step by step, follow this guide for implementing 1. EVM-Server 2. DAS-Pinner 3. Block Results Producer #### 1. Run EVM-Server The EVM-Server is a stateless EVM block execution tool. It's stateless because in Ethereum nodes like geth, it doesn't need to maintain database of blocks to do a block execution or re-execution. The `evm-server` service transforms Block Specimens to Block Results entirely on the input of underlying capture of Block Specimen data. Clone [covalenthq/erigon](https://github.com/covalenthq/erigon/) that contains the fork for this particular stateless execution function, Current stable version is `v1.3.2-evm`, build the `evm-server` binary ```shell theme={null} $ make evm ``` ```shell theme={null} $ ./build/bin/evm t8n --server.mode [INFO] [06-22|13:53:54.148] Listening port=3002 ``` **Note:** `evm-server` occupies the port 3002 on your local at `http://127.0.0.1:3002`", so make sure this port in not occupied by any other service prior to start. #### 2. Run DAS-Pinner DAS-Pinner serves as the interface to the storage layer of the network. It's a lightweight IPFS service that stores and pins data on the IPFS network using web3.storage. It is designed to work seamlessly with the DAS Light-Client, enabling the retrieval of data from the IPFS network and its verification using the DAS protocol.
It is meant for uploading/fetching network artifacts (like block specimens and block results or any other processed block data) files of the Covalent Decentralized Network. Clone [covalenthq/ewm-das](https://github.com/covalenthq/ewm-das/) and build the pinner server binary ```shell theme={null} $ git clone https://github.com/covalenthq/ewm-das.git $ cd ewm-das $ git checkout tags/v0.17.6 ## update if newer release is announced $ ./install-trusted-setup.sh $ make build-pinner ``` `ipfs-pinner` by getting `W3_AGENT_KEY` & `Proof.out` from web3.storage and adding it to an `.envrc` file. ```shell theme={null} $ cat .envrc export W3_AGENT_KEY="<>" export W3_DELEGATION_FILE="<>" ``` You should get the "w3 agent key" and "delegation proof file" from Covalent. After that, start the `ipfs-pinner` server. ```shell theme={null} ./bin/pinner -w3-agent-key $W3_AGENT_KEY -w3-delegation-file $W3_DELEGATION_FILE generating 2048-bit RSA keypair...done peer identity: QmZQSGUEVKQuCmChKqTBGdavEKGheCQgKgo2rQpPiQp7F8 Computing default go-libp2p Resource Manager limits based on: - 'Swarm.ResourceMgr.MaxMemory': "17 GB" - 'Swarm.ResourceMgr.MaxFileDescriptors': 61440 Applying any user-supplied overrides on top. Run 'ipfs swarm limit all' to see the resulting limits. 2023/04/20 12:47:49 Listening... ``` Note: das-pinner occupies the port 5080 on your local at [http://127.0.0.1:5080/api/v1](http://127.0.0.1:5080/api/v1), so make sure this port in not occupied by any other service prior to start. #### Debugging: Repo Migrations In case of the following error ```shell theme={null} 2023-03-10T08:46:18.621-0800 FATAL ipfs-pinner ipfs-pinner/pinner.go:29 error initializing ipfs node: ipfs repo needs migration, please run migration tool. See https://github.com/ipfs/fs-repo-migrations/blob/master/run.md Sorry for the inconvenience. In the future, these will run automatically. ``` Run the following migrations ```shell theme={null} wget https://dist.ipfs.tech/fs-repo-migrations/v2.0.2/fs-repo-migrations_v2.0.2_darwin-arm64.tar.gz tar -xvf fs-repo-migrations_v2.0.2_darwin-arm64.tar.gz cd fs-repo-migrations/ ./fs-repo-migrations ``` #### 3. Run Block Results Producer (BRP): BRP is the primary orchestrator and supervisor for all transformation pipeline processes that locates a source Covalent Network data object to apply a tracing/execution/transformational rule to and outputs a new object generated from using such a rule. Both the source and output are available through a decentralized storage service such as a wrapped IPFS node. A transformation-proof transaction is emitted, confirming that it has done this work along with the output content ids (ipfs) access URL. To define what these three components are: **Source:** The Block Specimen that serves as an input to the Refiner. Proof transactions made earlier to a smart contract with the respective cids (content ids) are where the source is checked. **Rule:** A transformation plugin (or server) that can act on the Block Specimen (source). These can be compared to blueprints that have been shown to produce the data objects needed. Furthermore, anyone can create these rules to get a desired data object. Rule (or server) versions thus need to exist, tied to the decoded block specimen versions they are applied on. **Target:** The output generated from running the rule over the object that came from the source that is the block result. #### Steps to Run BRP: Clone the `refiner` repo ```shell theme={null} git clone https://github.com/covalenthq/refiner.git cd refiner git checkout tags/v0.6.1 ``` Get your `BLOCK_RESULT_OPERATOR_PRIVATE_KEY` that has `GLMR` tokens for Moonbeam and is already whitelisted as Block Result Producer operator. Set the following environment variables for the local refiner by creating an `.envrc.local` file * Copy paste the environment variables into this file ```shell​ theme={null} touch .envrc.local export BLOCK_RESULT_OPERATOR_PRIVATE_KEY="BRP-OPERATOR-PK-WITHOUT-0x-PREFIX" export NODE_ETHEREUM_MAINNET="<>" export IPFS_PINNER_URL="http://127.0.0.1:5080/api/v1" export EVM_SERVER_URL="http://127.0.0.1:3002" ``` Call to load `.envrc.local + .envrc` files with the command below and observe the following output, make sure the environment variables are loaded into the shell.
* Once the env vars are passed into the `.envrc.local` file and loaded in the shell with `direnv allow .`, build the `refiner` application for the `prod` env i.e moonbeam mainnet or `dev` env for moonbase alpha as discussed before. ```shell theme={null} $ direnv allow . #make sure you see these being loaded direnv: loading ~/refiner/.envrc direnv: loading ~/refiner/.envrc.local direnv: export +BLOCK_RESULT_OPERATOR_PRIVATE_KEY +EVM_SERVER_URL +IPFS_PINNER_URL +NODE_ETHEREUM_MAINNET +W3_AGENT_KEY +W3_DELEGATION_FILE ``` Get all the required dependencies and build the `refiner` app. ```shell theme={null} mix local.hex --force && mix local.rebar --force && mix deps.get ``` * For moonbeam mainnet. * For moonbase alpha. ```shell theme={null} #moonbeam-mainnet MIX_ENV=prod mix release ​ #moonbase-alpha MIX_ENV=dev mix release ​ . .. .... evm-server: http://127.0.0.1:3002 ipfs-node: http://127.0.0.1:5080/api/v1 * assembling refiner-0.2.12 on MIX_ENV=dev * skipping runtime configuration (config/runtime.exs not found) * skipping elixir.bat for windows (bin/elixir.bat not found in the Elixir installation) * skipping iex.bat for windows (bin/iex.bat not found in the Elixir installation) Release created at _build/dev/rel/refiner    # To start your system    _build/dev/rel/refiner/bin/refiner start Once the release is running:    # To connect to it remotely    _build/dev/rel/refiner/bin/refiner remote    # To stop it gracefully (you may also send SIGINT/SIGTERM)    _build/dev/rel/refiner/bin/refiner stop To list all commands:   _build/dev/rel/refiner/bin/refiner ``` Start the `refiner` application and execute the proof-chain block specimen listener call which should run the Refiner pipeline pulling Block Specimens from IPFS using the cids read from recent proof-chain finalized transactions, decoding them, and uploading and proofing Block Results while keeping a track of failed ones and continuing (soft real-time) in case of failure. The erlang concurrent fault tolerance allows each pipeline to be an independent worker that can fail (for any given Block Specimen) without crashing the entire pipeline application. Multiple pipeline worker children threads continue their work in the synchronous queue of Block Specimen AVRO binary files running the stateless EVM binary (`evm-server`) re-execution tool. ```shell theme={null} #moonbeam-mainnet MIX_ENV=prod mix run --no-halt --eval 'Refiner.ProofChain.BlockSpecimenEventListener.start()'; ​ #moonbase-alpha MIX_ENV=dev mix run --no-halt --eval 'Refiner.ProofChain.BlockSpecimenEventListener.start()'; ​ ​ .. ... refiner       | [info] found 1 bsps to process ipfs-pinner | 2023/06/29 20:28:30 unixfsApi.Get: getting the cid: bafybeiaxl44nbafdmydaojz7krve6lcggvtysk6r3jaotrdhib3wpdb3di ipfs-pinner | 2023/06/29 20:28:30 trying out https://w3s.link/ipfs/bafybeiaxl44nbafdmydaojz7krve6lcggvtysk6r3jaotrdhib3wpdb3di ipfs-pinner | 2023/06/29 20:28:31 got the content! refiner       | [info] Counter for ipfs_metrics - [fetch: 1] refiner       | [info] LastValue for ipfs_metrics - [fetch_last_exec_time: 0.001604] refiner       | [info] Sum for ipfs_metrics - [fetch_total_exec_time: 0.001604] refiner       | [info] Summary for ipfs_metrics - {0.001604, 0.001604} refiner       | [debug] reading schema `block-ethereum` from the file /app/priv/schemas/block-ethereum.avsc refiner       | [info] Counter for bsp_metrics - [decode: 1] refiner       | [info] LastValue for bsp_metrics - [decode_last_exec_time: 0.0] refiner       | [info] Sum for bsp_metrics - [decode_total_exec_time: 0.0] refiner       | [info] Summary for bsp_metrics - {0.0, 0.0} refiner       | [info] submitting 17586995 to evm http server... evm-server   | [INFO] [06-29|20:28:31.859] input file at                           loc=/tmp/3082854681 evm-server   | [INFO] [06-29|20:28:31.862] output file at:                         loc=/tmp/1454174090 evm-server   | [INFO] [06-29|20:28:32.112] Wrote file                               file=/tmp/1454174090 refiner       | [info] writing block result into "/tmp/briefly-1688/briefly-576460747542186916-YRw0mRjfExGMk4M672" refiner       | [info] Counter for bsp_metrics - [execute: 1] refiner       | [info] LastValue for bsp_metrics - [execute_last_exec_time: 3.14e-4] refiner       | [info] Sum for bsp_metrics - [execute_total_exec_time: 3.14e-4] refiner       | [info] Summary for bsp_metrics - {3.14e-4, 3.14e-4} ipfs-pinner | 2023/06/29 20:28:32 generated dag has root cid: bafybeic6ernzbb6x4qslwfgklveisyz4vkuqhaafqzwlvto6c2njonxi3e ipfs-pinner | 2023/06/29 20:28:32 car file location: /tmp/249116437.car [119B blob data] ipfs-pinner | 2023/06/29 20:28:34 Received /health request: source= 127.0.0.1:34980 status= OK ipfs-pinner | 2023/06/29 20:28:34 uploaded file has root cid: bafybeic6ernzbb6x4qslwfgklveisyz4vkuqhaafqzwlvto6c2njonxi3e refiner       | [info] Counter for ipfs_metrics - [pin: 1] refiner       | [info] LastValue for ipfs_metrics - [pin_last_exec_time: 0.002728] refiner       | [info] Sum for ipfs_metrics - [pin_total_exec_time: 0.002728] refiner       | [info] Summary for ipfs_metrics - {0.002728, 0.002728} refiner       | [info] 17586995:48f1e992d1ac800baed282e12ef4f2200820061b5b8f01ca0a9ed9a7d6b5ddb3 has been successfully uploaded at ipfs://bafybeic6ernzbb6x4qslwfgklveisyz4vkuqhaafqzwlvto6c2njonxi3e refiner       | [info] 17586995:48f1e992d1ac800baed282e12ef4f2200820061b5b8f01ca0a9ed9a7d6b5ddb3 proof submitting refiner       | [info] Counter for brp_metrics - [proof: 1] refiner       | [info] LastValue for brp_metrics - [proof_last_exec_time: 3.6399999999999996e-4] refiner       | [info] Sum for brp_metrics - [proof_total_exec_time: 3.6399999999999996e-4] refiner       | [info] Summary for brp_metrics - {3.6399999999999996e-4, 3.6399999999999996e-4} refiner       | [info] 17586995 txid is 0xd8a8ea410240bb0324433bc26fdc79d496ad0c8bfd18b60314a05e3a0de4fb06 refiner       | [info] Counter for brp_metrics - [upload_success: 1] refiner       | [info] LastValue for brp_metrics - [upload_success_last_exec_time: 0.0031149999999999997] refiner       | [info] Sum for brp_metrics - [upload_success_total_exec_time: 0.0031149999999999997] refiner       | [info] Summary for brp_metrics - {0.0031149999999999997, 0.0031149999999999997} refiner       | [info] Counter for refiner_metrics - [pipeline_success: 1] refiner       | [info] LastValue for refiner_metrics - [pipeline_success_last_exec_time: 0.0052] ``` Check logs for any errors in the pipeline process and note the performance metrics in line with execution. Checkout the documentation on what is being measured and why [here](https://github.com/covalenthq/refiner/blob/main/docs/METRICS.md/). ```shell theme={null} tail -f logs/log.log .. ... refiner       | [info] Counter for refiner_metrics - [pipeline_success: 1] refiner       | [info] LastValue for refiner_metrics - [pipeline_success_last_exec_time: 0.0052] refiner       | [info] Sum for refiner_metrics - [pipeline_success_total_exec_time: 0.0052] refiner       | [info] Summary for refiner_metrics - {0.0052, 0.0052} ``` Alternatively - check proof-chain logs for correct block result proof submissions and transactions made by your block result producer. [For moonbeam](https://moonscan.io/address/0x4932bDc983e5146224b9C2e68cfFBFEb004A2824). [For moonbase](https://moonbase.moonscan.io/address/0xCBC44F143FB5baf26e45FB6C7A4fC13e6ca0fa09). **Note:** For any issues associated with building and re-compiling execute the following commands, that cleans, downloads and re-compiles the dependencies for refiner. ```shell theme={null} rm -rf _build deps && mix clean && mix deps.get && mix deps.compile ``` If you got everything working so far. Congratulations! You're now a Refiner operator on the CXT Network. Set up Grafana monitoring and alerting from links in the [additional resources](https://github.com/covalenthq/refiner#additional-resources) section. #### Troubleshooting To avoid permission errors with `~/.ipfs` folder execute the following in your home directory. ```shell theme={null} sudo chmod -R 770 ~/.ipfs ``` To avoid netscan issue execute the following against ipfs binary application. ```shell theme={null} sudo chmod -R 700 ~/.ipfs ipfs config profile apply server ``` To avoid issues with ipfs-pinner v1.1.0, a small repo migration for the local `~/.ipfs` directory may be needed from - [here](https://github.com/ipfs/fs-repo-migrations/blob/master/run.md). For linux systems follow the below steps. ```shell theme={null} wget https://dist.ipfs.tech/fs-repo-migrations/v2.0.2/fs-repo-migrations_v2.0.2_linux-amd64.tar.gz tar -xvf fs-repo-migrations_v2.0.2_linux-amd64.tar.gz cd fs-repo-migrations chmod +x ./fs-repo-migrations ./fs-repo-migrations ``` #### Bugs Reporting Contributions Please follow the guide in docs [contribution guidelines](https://github.com/covalenthq/refiner/blob/main/docs/CONTRIBUTING.md) for bug reporting and contributions. #### Scripts In order to run the Refiner docker compose services as a service unit. The example service unit file in [docs](https://github.com/covalenthq/refiner/blob/main/docs/refiner-compose.service) should suffice. After adding the env vars in their respective fields in the service unit file, enable the service and start it. ```shell theme={null} sudo systemctl enable refiner-compose.service sudo systemctl start refiner-compose.service ``` **Note::** To run docker compose as a non-root user for the above shown service unit, you need to create a docker group (if it doesn't exist) and add the user “blockchain” to the docker group. ```shell theme={null} sudo groupadd docker sudo usermod -aG docker blockchain sudo su - blockchain docker run hello-world ``` #### Appendix #### Run With Existing IPFS-Pinner Service On a system where an `ipfs-pinner` instance is already running, use this modified `.envrc.local` and `docker-compose-mbeam.yml` ```shell theme={null} export BLOCK_RESULT_OPERATOR_PRIVATE_KEY="BRP-OPERATOR-PK-WITHOUT-0x-PREFIX" export NODE_ETHEREUM_MAINNET="<>" export IPFS_PINNER_URL="http://host.docker.internal:5080/api/v1" export EVM_SERVER_URL="http://evm-server:3002" ``` **Note:** When passing the private key into the env vars as above please remove the `0x` prefix so the private key env var has exactly 64 characters. ```shell theme={null} version: '3' # runs the entire refiner pipeline with all supporting services (including refiner) in docker # set .env such that all services in docker are talking to each other only; ipfs-pinnern is assumed # to be hosted on the host machine. It's accessed through http://host.docker.internal:5080//api/v1 url from # inside refiner docker container. services: evm-server: image: "us-docker.pkg.dev/covalent-project/network/evm-server:stable" container_name: evm-server restart: always labels: "autoheal": "true" expose: - "3002:3002" networks: - cxt-net ports: - "3002:3002" refiner: image: "us-docker.pkg.dev/covalent-project/network/refiner:stable" container_name: refiner links: - "ewm-das:ewm-das" - "evm-server:evm-server" # build: # context: . # dockerfile: Dockerfile restart: always depends_on: ewm-das: condition: service_healthy evm-server: condition: service_healthy entrypoint: > /bin/bash -l -c " echo "moonbeam-node:" $NODE_ETHEREUM_MAINNET; echo "evm-server:" $EVM_SERVER_URL; echo "ewm-das:" $IPFS_PINNER_URL; cd /mix; MIX_ENV=prod mix deps.get; MIX_ENV=prod mix release --overwrite; MIX_ENV=prod mix run --no-halt --eval 'Refiner.ProofChain.BlockSpecimenEventListener.start()';" environment: - NODE_ETHEREUM_MAINNET=${NODE_ETHEREUM_MAINNET} - BLOCK_RESULT_OPERATOR_PRIVATE_KEY=${BLOCK_RESULT_OPERATOR_PRIVATE_KEY} - EVM_SERVER_URL=${EVM_SERVER_URL} - IPFS_PINNER_URL=${IPFS_PINNER_URL} networks: - cxt-net ports: - "9568:9568" autoheal: image: willfarrell/autoheal container_name: autoheal volumes: - "/var/run/docker.sock:/var/run/docker.sock" environment: - AUTOHEAL_INTERVAL=10 - CURL_TIMEOUT=30 networks: cxt-net: ``` and start the `refiner` and `evm-server` services: ```shell theme={null} $ docker compose -f "docker-compose-mbeam.yml" up --remove-orphans [+] Running 3/3 ⠿ Network refiner_cqt-net Created 0.0s ⠿ Container evm-server Started 0.7s ⠿ Container refiner Started 1.5s ```