Overview

Bakkt provides clients with a durable Event Queue to which they can connect and receive updates on cryptocurrency orders, transfers, batch trade and allocations, and gifting rewards. This implementation leverages the Amazon Web Service (AWS) Simple Queue Service (SQS).

Many of the Bakkt APIs rely on workflows with event notifications to client applications. The following information provides specific aspects of the Event Queue implementation.

Core Concepts and Functionality

The Bakkt AWS SQS provides an easy and reliable means of message transmission. The Amazon Simple Queue Service provides the basic AWS documentation to assist in implementing the Bakkt Event Queue. The mechanics of the SQS ensure the reliability of event delivery. Bakkt retains events sent to the queue for five days or when the system acknowledges receipt of the event. As part of the process of receiving events from the SQS, clients must acknowledge when events successfully process to ensure the system delivers all events at least once. Clients can use the following workflow to implement the Event Queue.

Workflow

Connect to the Event Queue

Client systems must authenticate to the Bakkt AWS SQS site using the same credentials they use for their AWS S3 document management system as explained in the AWS Credential Setup instructions. Bakkt recommends that clients use long polling to know when they have a message to process.

Bakkt provides clients with queue URLs to find and receive messages. Bakkt provides a queue URL in each of its DR failover regions. The QueueUrl definition from AWS is the URL of the Amazon SQS, and the type is String. Clients can see the Disaster Recovery (DR) section below for best-practice information.

Receive and Acknowledge Messages from the Event Queue

When messages are available in the queue (via long polling), clients must receive and process the messages from the Event Queue. SQS protocol dictates that clients process messages in their applications and then delete the messages from the SQS Event Queue to acknowledge when they receive and process the messages. The AWS documentation explains this process in SQS.

Event Queue Notification Message

The Bakkt Event Queue does not utilize all available fields in the SQS message protocol. The body of the SQS messages includes all of the necessary Bakkt notification information. Client systems do not have to use any other available fields in the SQS message. The following fields are available in Bakkt Event Notifications.

Field	Type	Description
`message_id`	UUID	Each message has a unique identifier (ID) when Bakkt sends it to the Event Queue; clients use this ID when they receive messages to determine if they already processed them Bakkt strives to send each message only one time; however, there are certain race conditions where the system may send duplicate messages, so clients should build applications to prevent duplicate processing of the same events
`event_type`	String	This field is in the structured format for the message that indicates the domain, schema, and version; the domain tells clients the Bakkt system sending the event Current domains include, but are not limited to, orders, transfers, accounts, and allocation; the schema provides the structure of the message; each event domain has its own set of schemas in the open API specification; the version is the open API specification version NOTE: Bakkt supports backwards compatibility; as versions of a schema change, the system can send multiple versions of the same message
`timestamp`	String	This field is the timestamp for event publishing; this is not for event triggering
`payload`	Object	This object is the event object for the specific event notification

The following field is not present in the message body, but clients may retrieve it from message attributes.

Field	Type	Description
`message_group_id`	String	The Message Group ID assigns the partition key to preserve ordering of the messages within that key Some example use cases are: Client Account ID - Many of the Bakkt systems perform actions related to an investor account number; in these applications, the account is the `message_group_id` to allow the Bakkt client to better order investor account-related events. Client Order ID - The order ID the client system assigns to the submitted order (the system may add other group ids in the future)

Example
See the following sample event (with sample payload).

{
  "event_type": "transfers/transfer/1",
  "message_id": "f7c0b3a3-33f0-40ae-af6f-db9afaf2017b",
  "payload": {
    "status": {
      "account": "9YN05004",
      "blockchainHash": "428d4bf6e349b3647ee541139ceecbcf…",
      "createdAt": "2022-06-14T17:35:05.940Z",
      "currency": "BTC",
      "id": "db990ff4-b3aa-4e1e-96e2-9d4acd33334b",
      "modifiedAt": "2022-06-14T17:35:06.783Z",
      "quantity": "0.0001",
      "receiveAddress": "2N4JJ7cC55T8MongLqj4B9LrW7doG1DLndM",
      "status": "COMPLETE",
      "type": "DEPOSIT",
      "version": 8
    }
  },
  "timestamp": "2022-06-14T17:35:06.849810Z"
}

Disaster Recover and Event Queues

Each AWS region presents a single SQS queue—one for primary use and one for Bakkt's disaster recovery footprint. The status update events are available in each queue with a standard envelope wrapping each event to describe the event type. The SQS event stream does not include market data events due to their volume and non-durable nature.

🚧
Primary and Disaster Recovery Queues
Bakkt highly recommends that clients consume both the primary and disaster recovery queues at all times. The Event system removes events after clients receive them which brings both queues into a contemporary state. Since the system publishes all events to both queues, the process must use the deduplication identifier to ensure that the system only processes events once. If clients only consume the primary queue, they must receive and retain five days worth of events in case of region failovers.

Connecting to the AWS SQS

Clients can use the following process to connect to the Amazon Web Service Simple Queue Service (AWS SQS).

Notice: Some events are not enabled by default. If you need any particular events, please let us know.

Workflow

Use the standard AWS options to allow authentication using the typical environment variables.

func example() {
  logger := log.Default()
  
  awsSessionOptions := session.Options{
    SharedConfigState: session.SharedConfigEnable,
  }
  awsSession := session.Must(session.NewSessionWithOptions(awsSessionOptions))

Create the SQS client.

sqsService := sqs.New(awsSession, aws.NewConfig())

  queueUrl :=
"https://sqs.us-east-2.amazonaws.com/{AWSID}/uat-{corr}-events.fifo"

Define the parameters for the ReceiveMessage call. The WaitTimeSeconds option enables long-polling (which Bakkt recommends).

for {
    sqsReceiveMessageInput := sqs.ReceiveMessageInput{
        QueueUrl: &queueUrl,
        WaitTimeSeconds: aws.Int64(10),
    }

Receive message(s) from the AWS SQS queue.

sqsReceiveMessageOutput, err :=
sqsService.ReceiveMessage(&sqsReceiveMessageInput)
       if err != nil {
           logger.Fatal(err)
       }

Loop through each received message. The array is empty if no messages are available. Extract the body from the SQS message and unmarshal the SQS message body.

for _, sqsMessage := range sqsReceiveMessageOutput.Messages {

    sqsMessageBody := sqsMessage.Body
    
    unMarshalledBody := make(map[string]interface{})
    err := json.Unmarshal([]byte(*sqsMessageBody), &unMarshalledBody)
    if err != nil {
        log.Fatal(err)
    }

Process the message. In the following example, Bakkt prints the body as JSON.

formattedBody, err := json.MarshalIndent(unMarshalledBody, "", " ")
if err != nil {
    log.Fatal(err)
}
fmt.Println(string(formattedBody))

Define the parameters for the DeleteMessage call. The third line contains a unique reference to delete a received SQS message.

sqsDeleteMessageInput := sqs.DeleteMessageInput{
    QueueUrl: &queueUrl,
    ReceiptHandle: sqsMessage.ReceiptHandle,
}

Acknowledge that the message processes by deleting it from the SQS queue.

_, err = sqsService.DeleteMessage(&sqsDeleteMessageInput)
      if err != nil {
        logger.Fatal(err)
      }
    }
  }
}

Sample Events

Order Entry Examples

{
  "message_id": "896b13e3-1c01-4bbc-a471-759787aa1a9b",
  "event_type": "orders/order/1",
  "timestamp": "2022-06-29T06:14:32.218047552Z",
  "payload": {
    "eventTime": "2022-06-29T06:14:32.203Z",
    "id": "2e8d4430-c89a-4401-9970-a27ba457a4ad",
    "order": {
      "averagePrice": 0,
      "cancels": null,
      "createdAt": "2022-06-29T06:14:32.175Z",
      "cumulativeQuantity": 0,
      "executions": [],
      "id": "7a10224f-be15-4cd5-8cff-354f2051468a",
      "leavesQuantity": 1,
      "request": {
        "account": "9YN05004",
        "clientOrderId": "7a10224f-be15-4cd5-8cff-354f2051468a",
        "notional": 1,
        "side": "BUY",
        "symbol": "LINKUSD",
        "timeInForce": "IOC",
        "type": "MARKET"
      },
      "status": "IN_PROGRESS",
      "working": true
    }
  }
}

{
  "event_type": "orders/order/1",
  "message_id": "11d00905-76f2-44ec-9f1e-7d042dbcbab3",
  "payload": {
    "eventTime": "2022-07-05T06:02:49.919Z",
    "id": "2e8d4430-c89a-4401-9970-a27ba457a4ad",
    "order": {
      "averagePrice": "20600",
      "cancels": null,
      "createdAt": "2022-07-05T06:02:49.469Z",
      "cumulativeQuantity": "0.00005",
      "executions": [
        {
          "commission": "0.98",
          "executedAt": "2022-07-05T06:02:49.854Z",
          "id": "22070544000004442",
          "netMoney": "2.01",
          "price": "20504.49494949",
          "quantity": "0.00005",
          "side": "BUY",
          "symbol": "BTCUSD",
          "tradeDate": "2022-07-04"
        }
      ],
      "id": "2e8d4430-c89a-4401-9970-a27ba457a4ad",
      "leavesQuantity": "0",
      "request": {
        "account": "9YN05004",
        "clientOrderId": "ac8430b6-c6be-4ee6-a6d2-08593b5a2207",
        "price": "21659.27",
        "quantity": "0.000050",
        "side": "BUY",
        "symbol": "BTCUSD",
        "timeInForce": "DAY",
        "type": "LIMIT"
      },
      "status": "COMPLETE",
      "working": false
    }
  }
}

{
  "message_id": "07706ab0-27a1-46a9-be3d-268b81f5fe54",
  "event_type": "orders/order/1",
  "timestamp": "2022-06-29T06:15:19.369156922Z",
  "payload": {
    "eventTime": "2022-06-29T06:15:19.353Z",
    "id": "07706ab0-27a1-46a9-be3d-268b81f5fe54",
    "order": {
      "cancels": null,
      "createdAt": "2022-06-29T06:15:19.331Z",
      "executions": [],
      "id": "07706ab0-27a1-46a9-be3d-268b81f5fe54",
      "leavesQuantity": 0,
      "rejectReason": "unknown: empty quotes",
      "request": {
        "account": "9YN05004",
        "clientOrderId": "93e2c9f7-650c-4088-ad24-5f389f402483",
        "notional": 1,
        "side": "BUY",
        "symbol": "LUNAUSD",
        "timeInForce": "IOC",
        "type": "MARKET"
      },
      "status": "COMPLETE",
      "working": false
    }
  }
}

{
  "event_type": "orders/order/1",
  "message_id": "12cfa2dc-56b9-4303-a043-a07764b1dc71",
  "payload": {
    "eventTime": "2022-06-29T06:15:57.579Z",
    "id": "12cfa2dc-56b9-4303-a043-a07764b1dc71",
    "order": {
      "averagePrice": "0",
      "cancels": null,
      "createdAt": "2022-06-29T06:15:57.449Z",
      "cumulativeQuantity": "0",
      "executions": [],
      "id": "12cfa2dc-56b9-4303-a043-a07764b1dc71",
      "leavesQuantity": "1",
      "request": {
        "account": "9YN05004",
        "clientOrderId": "8aebd078-32b7-4b7f-9a6a-243b34e2a6b5",
        "notional": "1",
        "side": "BUY",
        "symbol": "MATICUSD",
        "timeInForce": "IOC",
        "type": "MARKET"
      },
      "status": "COMPLETE",
      "working": false
    }
  },
  "timestamp": "2022-06-29T06:15:57.595353048Z"
}

Transfer Event Examples

{
 "event_type": "transfers/transfer/1",
 "message_id": "2d0b0aad-6158-4c6f-b8a6-f45afc89f58c",
 "payload": {
   "status": {
     "account": "9YN05004",
     "clientTransactionId": "695d9a31-5010-4ff1-a54fb0df40c19e95",
     "createdAt": "2022-06-20T07:03:49.314Z",      
     "currency": "LTC",
     "id": "49dc1c41-e82c-41df-aa78-39228d117b5d",
     "modifiedAt": "2022-06-20T07:13:03.701Z",
     "quantity": "0.09575688073394495",
     "receiveAddress":"mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt",
     "status": "ACCEPTED",
     "type": "WITHDRAWAL",
     "version": 3
   }
 },
 "timestamp": "2022-06-20T07:13:03.707502672Z"
}

{
 "event_type": "transfers/transfer/1",
 "message_id": "dbe7dc6c-1447-437f-9dae-03043f6b44a7",
 "payload": {
   "status": {
     "account": "9YN05004",
     "clientTransactionId": "6a8ed54d-217e-4c0d-8fe8-8f33dffab23b",
     "createdAt": "2022-06-20T07:03:43.079Z",
     "currency": "BTC",
     "id": "5f9945be-174a-4f0a-ae28-65234ec23375",
     "modifiedAt": "2022-06-20T07:03:43.137Z",
     "quantity": "0.0001",
     "receiveAddress":"mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt",
     "status": "REJECTED",
     "type": "WITHDRAWAL",
     "version": 1
   }
 },
 "timestamp": "2022-06-20T07:03:43.144058655Z"
}

{
    "message_id": "a1524ac1-b8de-41ab-b2c5-bd25654afd8d",
    "event_type": "transfers/transfer/1",
    "timestamp": "2024-02-01T11:01:35.682060802Z",
    "payload": {
        "status": {
            "account": "9ZK05019",
            "blockchainHash": "88943c5a260a41db9d9e09fc80df100ce8704cd19383ea7f6d4081dcd26f89ef",
            "createdAt": "2024-02-01T11:01:35.482Z",
            "currency": "LTC",
            "id": "97a7ee1d-8c40-4e23-b208-875ad28239c8",
            "modifiedAt": "2024-02-01T11:01:35.674Z",
            "quantity": "0.05",
            "receiveAddress": "QUVBeDZL7ySosrskzMXNRGpkK9Knbns5fy",
            "status": "IN_PROGRESS",
            "statusReason": "awaiting confirmations further processing",
            "type": "DEPOSIT",
            "version": 3
        }
    }
}

{
    "message_id": "3cdef2e6-47b6-4b3b-9903-4fd3dd943dd6",
    "event_type": "transfers/transfer/1",
    "timestamp": "2024-02-01T11:07:26.123000518Z",
    "payload": {
        "status": {
            "account": "9ZK05019",
            "blockchainHash": "88943c5a260a41db9d9e09fc80df100ce8704cd19383ea7f6d4081dcd26f89ef",
            "createdAt": "2024-02-01T11:01:35.482Z",
            "currency": "LTC",
            "id": "97a7ee1d-8c40-4e23-b208-875ad28239c8",
            "modifiedAt": "2024-02-01T11:07:26.118Z",
            "quantity": "0.05",
            "receiveAddress": "QUVBeDZL7ySosrskzMXNRGpkK9Knbns5fy",
            "status": "COMPLETE",
            "type": "DEPOSIT",
            "version": 15
        }
    }
}

{
    "message_id": "4afa7a72-0125-4f0f-b7ca-da036c3c77af",
    "event_type": "transfers/transfer/1",
    "timestamp": "2024-02-01T11:07:26.270269021Z",
    "payload": {
        "status": {
            "account": "9ZK05019",
            "blockchainHash": "88943c5a260a41db9d9e09fc80df100ce8704cd19383ea7f6d4081dcd26f89ef",
            "clientTransactionId": "803af830-f424-4da8-90ea-498307c6c68b",
            "createdAt": "2024-02-01T10:58:15.111Z",
            "currency": "LTC",
            "id": "6773b209-6e6e-4d69-ac0b-d32085930910",
            "modifiedAt": "2024-02-01T11:07:26.266Z",
            "quantity": "0.05",
            "receiveAddress": "QUVBeDZL7ySosrskzMXNRGpkK9Knbns5fy",
            "status": "COMPLETE",
            "type": "WITHDRAWAL",
            "version": 6
        }
    }
}

Gifting Event Examples

{
    "message_id": "fcc16f8b-0e31-410c-a21b-ac07d5d47f5f",
    "event_type": "allocation/gift/1",
    "timestamp": "2024-04-24T08:59:38.495322892Z",
    "payload": {
        "createdAt": "2024-04-24T08:59:38.184Z",
        "id": "7df25c2e-2b70-4e9a-9d73-912e4f06820f",
        "notional": "5",
        "price": "66552.23",
        "quantity": "0.00007513",
        "request": {
            "account": "9af05004-0000-0000-0000-000000000000",
            "clientTransactionId": "1fa82f64-5717-4562-b3fc-2c963f66afa6",
            "currency": "BTC",
            "notional": "5"
        },
        "status": "IN_PROGRESS"
    }
}

{
    "message_id": "a713dc3a-376b-4f04-91c1-1c2f03529137",
    "event_type": "allocation/gift/1",
    "timestamp": "2024-04-24T08:59:38.583876281Z",
    "payload": {
        "createdAt": "2024-04-24T08:59:38.184Z",
        "id": "7df25c2e-2b70-4e9a-9d73-912e4f06820f",
        "notional": "5",
        "price": "66552.23",
        "quantity": "0.00007513",
        "request": {
            "account": "9af05004-0000-0000-0000-000000000000",
            "clientTransactionId": "1fa82f64-5717-4562-b3fc-2c963f66afa6",
            "currency": "BTC",
            "notional": "5"
        },
        "status": "COMPLETE"
    }
}

{
 "event_type": "allocation/gift/1",
 "message_id": "56bdd377-e8e3-41ed-9dc1-bff4ced24395",
 "payload": {
     "createdAt": "2022-06-21T23:01:09.486Z",
     "id": "d0b55788-a12c-4090-bd48-bf07e568885b",
     "notional": "1",
     "price": "6.941",
     "quantity": "0.14407146",
     "request": {
       "account": "9YN05004",
       "clientTransactionId": "52874f45-ebeb-41cd-ae64-bff039857566",
       "currency": "ATOM",
       "notional": "1"  
     },
     "status": "REJECTED"
 },
 "timestamp": "2022-06-21T23:01:09.527566335Z"
}

Single Allocation Event Examples

{
 "message_id":"21dd4740-748b-4b98-a17a-4dee15ae1c14",
 "event_type":"allocation/inventory/1",
 "timestamp":"2022-06-22T15:52:09.500025743Z",
 "payload":{
    "createdAt":"2022-06-22T15:52:09.377Z",
    "id":"fa885902-080b-4c83-8060-81ff7e3ac82e",
    "request":{
       "account":"9af05004-0000-0000-0000-000000000000",
       "clientTransactionId":"650e3b63-cce2-4a58-a955-8e7deba98b9a",
       "currency":"BTC",
       "price":"21000",
       "quantity":"0.001",
       "side":"SELL"
    },
    "status":"IN_PROGRESS"
 }
}

{
 "message_id":"ac778a04-a5b2-4970-b016-e8a7ebd162da",
 "event_type":"allocation/inventory/1",
 "timestamp":"2022-06-22T15:52:09.771713974Z",
 "payload":{
    "createdAt":"2022-06-22T15:52:09.377Z",
    "id":"fa885902-080b-4c83-8060-81ff7e3ac82e",
    "request":{
       "account":"9af05004-0000-0000-0000-000000000000",
       "clientTransactionId":"650e3b63-cce2-4a58-a955-8e7deba98b9a",
       "currency":"BTC",
       "price":"21000",
       "quantity":"0.001",
       "side":"SELL"
    },
    "status":"COMPLETE"
 }
}

{
 "message_id":"621a8b4a-f290-4e28-8340-a36a61feff7b",
 "event_type":"allocation/inventory/1",
 "timestamp":"2022-06-22T18:46:45.864149907Z",
 "payload":{
    "createdAt":"2022-06-22T18:46:45.338Z",
    "id":"90f53778-0f7a-46de-a6b6-c48e0c42aa3c",
    "request":{
       "account":"9af05004-0000-0000-0000-000000000000",
       "clientTransactionId":"46b489f8-ed65-42ea-81a0-c2429f8944f8",
       "currency":"BTC",
       "price":"21000",
       "quantity":"10",
       "side":"BUY"
    },
    "status":"REJECTED"
 }
}

Batch Event Examples
See the following examples of batch events with simple payloads.

{
  "event_type": "allocation/batch/1",
  "message_id": "6f6ab118-5c02-41e1…",
  "payload": {
    "allocations": 0,
    "createdAt": "2022-10-03T06:02:13.417Z",
    "id": "5c394ed8-b7a3-48d8-b793-7ee6…",
    "request": {
      "batchTransactionId": "49136c79-3ce9-4f8d-8794...",
      "symbol": "BTCUSD",
      "type": "NOTIONAL"
    },
    "status": "OPEN"
  },
  "timestamp": "2022-10-03T06:02:13.424552751Z"
}

{
  "event_type": "allocation/batch/1",
  "message_id": "b8b55e4a-3637-42fd-80e5-e2311…",
  "payload": {
    "allocations": 1,
    "createdAt": "2022-10-03T06:02:16.121Z",
    "id": "3ac9077f-d7a6-4200-bc1f-669d567…",
    "request": {
      "batchTransactionId": "95875dd9-c380-4cb5… ",
      "symbol": "BTCUSD",
      "type": "NOTIONAL"
    },
    "status": "CLOSED"
  },
  "timestamp": "2022-10-03T06:02:16.190980550Z"
}

{
  "event_type": "allocation/batch/1",
  "message_id": "0c61649a-25db-424c-904e-5568b…",
  "payload": {
    "createdAt": "2022-06-24T14:33:31.605Z",
    "id": "458b1182-6449-48e1-b2c0-3c942c…",
    "netMoney": "200",
    "price": "21440.92",
    "quantity": "0.00932796",
    "request": {
      "account": "6a605008…",
      "amount": "200",
      "batchTransactionId": "d26q8619-7uc1-4fcf-99b6-…",
      "clientTransactionId": "69d4c791-4db3-486b-b4a8-d5fb…",
      "side": "BUY",
      "symbol": "BTCUSD"
    },
    "status": "COMPLETE"
  },
  "timestamp": "2022-06-24T14:34:16.749988161Z"
}

{
  "event_type": "allocation/batch/1",
  "message_id": "6f11b118-5c02-41e1…",
  "payload": {
    "allocations": 0,
    "createdAt": "2022-10-03T06:02:13.417Z",
    "id": "5c394ed8-b711-48d8-b793-7ee6…",
    "rejectReason": "Insufficient BCH available to allocate",
    "request": {
      "account": "AT00000",
      "symbol": "BCHUSD",
      "quantity": 0.0001,
      "type": "QUANTITY"
    },
    "batchTransactionId": "4911179-3ce9-4f8d-8794...",
    "clientTransactionId": "94c28222-1111-1111-a105-a359913c4c4d",
    "side": "SELL"
    "status": "REJECTED"
  },
  "timestamp": "2022-10-03T06:02:13.424552751Z"
}

Overview

Core Concepts and Functionality

Workflow

Connect to the Event Queue

Receive and Acknowledge Messages from the Event Queue

Event Queue Notification Message

Disaster Recover and Event Queues

🚧Primary and Disaster Recovery Queues

Connecting to the AWS SQS

Workflow

Sample Events

🚧
Primary and Disaster Recovery Queues