Scaling Transactions API with Time Buckets

Getting bulk transactions with time buckets

Covalent just launched a brand new GET Get bulk time bucket transactions for address (v3) endpoint that allows you to get all transactions in bulk, simply by specifying a time bucket. This is useful for developers who want to get all granular transactions data before, during, or after a specified time interval, especially for high-volume wallet addresses.

What are time buckets?

Time buckets are 15-min windows locked in time, beginning on 1 Jan 1970 00:00:00 (Unix Epoch time). Time bucket 0 starts at 1970-01-01 00:00:00 inclusive and ends at 1970-01-01 00:14:59 inclusive. When you specify a time bucket, all transactions within that 15-min window are returned.

Calculating time buckets

Suppose you want to find all transactions between 2023-03-08 03:35:00 GMT and 2023-03-08 03:58:00 GMT for a particular address.

To calculate the time bucket, the following method can be used.

1. Convert the start time 2023-03-08 02:35:00 GMT to its unix time – 1678242900

2. Divide the unix time by 900 (number of seconds in 15 minutes)

   1. 1678242900 / 900 = 1864714.3333

3. Floor this number to the nearest integer – 1864714

That's it! The time bucket to use is 1864714.

1. This will give you the 15-min window of 02:30:00 to 02:45:00 (as we've rounded the time bucket down).

2. To get the next 15 minutes (till 03:00:00), simply supply the next time-bucket 1864715.

To get all the transactions for an account with many transactions, multiple calls will be necessary as you will need to crawl all the time buckets. Thus, it is recommended that transactions are stored locally so that the same time-bucket and wallet combination isn't called more than once.

Why are time buckets needed?

The initial version of the transaction endpoint was built to show the last N transactions for a wallet for display purposes. But having rich, granular, historical transaction data proved to be much more useful for other advanced use cases like calculating risk scores and profit-loss statements to calculate your tax liabilities. These sorts of paginated historical transaction queries are particularly high-intensity on our backend.

The time bucket method will scale to wallets with millions of transactions and not time out.

Calling transactions with time buckets

The GET Get bulk time bucket transactions for address (v3) endpoint takes three path parameters: chainName, address and timeBucket:

When you call the API with a particular time bucket:

Bash

curl "https://api.covalenthq.com/v1/eth-mainnet/bulk/transactions/0x88e6...5640/1864685/"

A sample response is:

JavaScript

{
  "data": {
    "address": "0x95222290dd7278aa3ddd389cc1e1d165cc4bafe5",
    "updated_at": "2023-03-08T05:48:33.744864117Z",
    "next_update_at": "2023-03-08T05:53:33.744893744Z",
    "quote_currency": "USD",
    "chain_id": 1,
    "chain_name": "eth-mainnet",
    "complete": true,
    "links": {
      "prev": "https://api.covalenthq.com/v1/eth-mainnet/bulk/transactions/0x95222290dd7278aa3ddd389cc1e1d165cc4bafe5/1864714/",
      "next": "https://api.covalenthq.com/v1/eth-mainnet/bulk/transactions/0x95222290dd7278aa3ddd389cc1e1d165cc4bafe5/1864716/"
    },
    "items": [
      {...18 items},
      {...18 items},
      {...18 items},
      {...18 items},
      {...18 items}
    ]
  },
  "error": false,
  "error_message": null,
  "error_code": null
}

The most important keys in the response are:

• links.prev - the link to the previous non-empty time bucket

• links.next - the link to the next non-empty time bucket so that you don't have to crawl every single time bucket exhaustively

• complete - boolean if the time bucket is in the past and you can be sure that the bucket has been fully filled. If complete is false, it means that the bucket is still being filled and you can fetch this bucket again to get the latest transactions (if any.)

Supported Chains

The time bucket extension is supported on all chains through the Unified API.

Sample Code

JavaScript

// For calculating time bucket

var epochInMilli = Date.parse("01 Jan 1970 00:12:00 GMT");
var epoch = epochInMilli / 1000;

var bucket = Math.floor(epoch / 900);