Bank integration tutorial (open banking) with Bridge SDK
Introduction
This tutorial shows you how to connect a bank to a cloud based ACH network built using the Minka Ledger. For this purpose we will be implementing a bridge service which connects the bank’s core systems with the cloud Ledger. We will be using the @minka/bridge-sdk
library in order to make the integration process faster.
Like always when building a production-ready service, follow your usual best practices on application design and security.
The code is written using TypeScript, but you can also use JavaScript.
Here is a diagram that will help you understand the role of the Bridge in the overall architecture of the system.
Setup
Prerequisites
We will need the following tools in order to develop and run the demo. You may already have some of them installed from the Making cross-ledger payments.
This tutorial is an extension of the Making cross-ledger payments . If you have not already, please complete it first to get familiar with basic Ledger concepts. The following steps assume that you already have a bank setup which is done as part of the previous tutorial.
Node.js and npm
https://nodejs.org/en/download/
Minka CLI tools
https://www.npmjs.com/package/@minka/cli
After installing Node.js and npm, you can install Minka CLI tools by running the following command.
Docker
https://docs.docker.com/get-docker/
Ngrok can be used to host and test the API service.
https://ngrok.com/docs/getting-started
If you are on a Mac and have brew installed, you can just run the command below. Otherwise, check out the getting started docs above
Curl will be used to test the API, but you can also use Postman or another tool you prefer.
Sending money from account to account
Setting up the Bridge service
The simplest case of sending money between two banks is a direct account to account transfer.
This means that we know both the source and target account number and the corresponding banks. An account in this context represents any account whose balance is managed outside of the ledger also known as external accounts.
This includes checking and savings accounts as well as loan and credit card accounts. Of course, other types of products are also possible. You can find more details in the reference documentation.
We will now start implementing the Bridge component that we will use to communicate with the Ledger. For now, payment initiation is going to be performed using the CLI tool.
Setup
You will need to have the NodeJS
and npm
prerequisites installed locally.
First, we will create an empty nodeJS project with a src
directory.
Now that we have an empty node project, the next step is to install typescript
and ts-node
.
We can do that using npm
.
We will now initialize the TypeScript project:
Now, you should have a file tsconfig.json
make sure that "target"
field is "es2018"
, if not, please update it. This version is required for regular expressions in the extractor.ts
Now we can test if everything is setup correctly by creating a simple hello world route in src/main.ts
.
To test our code we need to start the service.
If you want to avoid having to restart the service after every change, you can install a tool like nodemon
that will monitor your project for changes and automatically restart the service.
To install nodemon
run:
Then you should run the application using:
Setting up Bridge SDK
To develop our bridge service, we will be using the @minka/bridge-sdk
library. It uses a PostgreSQL database so let’s start by setting it up. Let’s start by creating a docker-compose.yaml
file with the following contents:
We will use the following command to start our database:
To stop it, just stop press Ctrl + C. If you want to continue using the same terminal, you can run the same command and add the -d
flag to detach the container from the terminal. In that case, in order to stop the container, you will need to run docker compose down
. The docker commands need to be run in the same directory where docker-compose.yaml
is located, otherwise, you will need to use the -f filename
argument.
Now that we have the database up and running, let’s set up a simple bridge service with default endpoints. We will first need to install the @minka/bridge-sdk
and @minka/ledger-sdk
libraries.
Since the library takes care of almost everything for us, the only thing that is left for us to do is to configure it. We will do that by using environment variables and the envalid
library. To make things easier, we will store the environment variables in a .env
file and use the dotenv
library to load it. Let’s first install the libraries:
We will now create the src/config.ts
file and add the following contents:
We should also create the src/.env
file with all the necessary environment variables:
To get LEDGER_PUBLIC_KEY
value, execute:
If you used the CLI to generate bridge public and private keys, execute the following command to get BRIDGE_PUBLIC_KEY
and BRIDGE_SECRET_KEY
values
Now we can edit src/main.ts
to configure the Bridge SDK. The code is pretty self-explanatory, it loads the configuration for the service and uses it to build the ServerService
and ProcessorService
from @minka/bridge-sdk
. The ServerService
will create all the necessary endpoints that we need and accept processing requests from Ledger. The ProcessorService
will process those requests and notify Ledger of the results.
Bridge Interface
We will build our bridge in this tutorial by adding functionality as we need it, step by step. Just to have a high level overview of what we will get in the end, here are all the endpoints that we will implement:
Endpoint | Request | Description |
---|---|---|
POST /v2/credits | prepare | Called during two-phase commit when Bridge needs to prepare a credit Entry. The Bridge should check if the account exists, is active and do other checks necessary to ensure the transfer intent can proceed. |
POST /v2/credits/:handle/commit | commit | Called during two-phase commit when Bridge needs to commit a credit Entry. This request must succeed and is considered successful even if there is an error while processing. In that case, the Bank needs to fix the issue manually. |
POST /v2/credits/:handle/abort | abort | Called during two-phase commit when Bridge needs to abort a credit Entry. This request gets called if there is an issue while processing the transfer. It can not fail just like commit and must completely reverse the transfer. |
POST /v2/debits | prepare | Called during two-phase commit when Bridge needs to prepare a debit Entry. Same as for credit, but needs to also hold or reserve the necessary funds on source account. |
POST /v2/debits/:handle/commit | commit | Called during two-phase commit when Bridge needs to commit a debit Entry. Same as for credit. |
POST /v2/debits/:handle/abort | abort | Called during two-phase commit when Bridge needs to abort a debit Entry. Same as for credit, but potentially also needs to release the funds. |
PUT /v2/intents/:handle | update | Called during two-phase commit when the Intent state updates. This can be used to update the Intent in local database so the Bank knows when it is fully processed. |
As you can see, the Bridge side of the Ledger - Bridge interface consists of 3 main routes and 7 endpoints in total. We will explore them one at a time in the next few sections.
Next, we are going to create two adapters, they contain the only custom logic we need to implement to connect to our core. Let’s create them and only log incoming requests for now.
The log is here just so that it looks the same as the old tutorial, the library has its own logging which is disabled on this branch.
We are just logging that the request happened and suspending further processing. We will build out the methods one at a time, however, before we do that we will need a mock banking core and some helper functions to extract all the data we need from requests.
Simulate banking core
To simulate a banking core we will create a simple module that will hold balances of client accounts.
It will have similar high-level concepts as a real ledger, but is rudimentary and just for demonstration purposes.
In the real world this would be replaced by API calls to the actual core.
The internals of this module are not important so feel free to copy/paste it. The thing to note is that this is an in-memory mock ledger with some accounts set up in advance that will enable us to simulate different cases.
It exposes a few methods and will throw errors in case of eg. insufficient balance. The credit
and debit
methods credit and debit client accounts respectively. The hold
method will hold client balances so that they can’t be spent and release
will make them available for spending. We will later use the preconfigured accounts in various scenarios.
Extracting request data
To extract data from requests, we will use a couple of functions which we will put into src/extractor.ts
. Of course request data should be validated as well.
Processing a credit Entry
Now that we are set up, we can create an intent and we should see some requests coming from the Ledger. Here we will use the signer, wallet and bridge that we created in the previous tutorial. We will use the same mint signer, wallet and bridge that you created in the previous tutorial and we will start by registering the Bridge with the Ledger. It should already exist so we will just update the URL to point to us. Before that, however, you will need a public URL that Ledger can target. If you don’t have one, you can use ngrok.
Start by opening a dedicated terminal for ngrok and running the following command.
If this is not your first time running ngrok, it is still possible that your URL changed so please check because you may need to update it anyway.
Leave ngrok running and continue the tutorial in a new terminal. You should now have three terminal windows open, one for the Bridge service, another one for ngrok and a new one for CLI commands.
Next, run the following command, substituting mint for your own bridge. When prompted, update the Server
field and set it to <bridge URL> from the command above.
We will now finally create an intent using the CLI tool like this.
We should see the first Entry record in Bridge logs. In all of the methods we are implementing, you can log the context
to see more details or just context.entry
and context.command
to see the requests.
Processing requests
We see that we got a request to the POST /v2/credits endpoint.
This request is part of the Ledger two-phase commit protocol, please read more about it in About Intents.
The Bridge SDK does most of the heavy lifting for us:
- saving the request
- idempotency
- some errors
- validation of signatures
- notifying ledger of entry processing
- retries
- coordinating request processing
- process final intent updates
We will only need to write 6 methods to actually process the request in the banking core. It is possible to have both sync and async request handlers, but we will stick with sync for this tutorial.
We need to prepare for crediting the client account which involves things like checking that it exists, that it’s active and similar in the banking core.
In case of debit, we will also need to check that the client has enough funds and put a hold on those funds.
After a prepare succeeds, we are no longer allowed to fail on commit. At that point the responsibility to commit or abort the transfer is passed on to Ledger.
Credit Entry
The already existing code will always be grayed out in this tutorial and unless otherwise specified, you just need to add the new code in red.
Let’s start with updating the prepare credit function. We need to add the following code.
Now when we run Test 1
again, we should get:
Commit credit
We have successfully processed the prepare
request for a credit Entry and Ledger continued processing up to the point where we got POST /v2/credits/:handle/commit. Now we will need to implement the commit
request to continue.
We will use the same pattern as we did to process prepare so we will edit the commit credit function accordingly.
As you know from About Intents, we are not permitted to fail the commit
request so if something happens during processing, we just suspend processing for now.
For all intents and purposes, the Entry is already committed
when we get the commit
, it was just not yet processed by the Bank.
We can now run Test 1
again and we should get something like this.
At this point, Ledger will also update the intent with final status using PUT /v2/intents/:handle, but we don’t see that here because the SDK handles it for us.
Processing a debit Entry
Now we will do the same thing we did above, but for debit endpoints. Lets start with the prepare debit handler.
You can notice that processing for the prepare
action is a little more complex than that of the credit counterpart because we have to hold the funds.
If we run into issues, we set the state to failed
and notify Ledger which will initiate an abort. If we instead notify Ledger that we are prepared
, we can no longer fail when we get the commit
action.
Prepare debit test
To test the code above, we will create a new intent that will send money from account:1@mint
to account:73514@tesla
.
We should now be seeing something like this.
Commit debit
We will now implement the commit debit endpoint.
Again, you can see that the processing for commit
debit is a little more complex than for commit
credit because we first need to release the funds and then execute a debit operation. Depending on how transfer orders are implemented, it is possible that in an real core this may be a single operation.
After running Test 2
again, we should get the following.
Combining credits and debits
We are now ready to play with our new Bridge 🥳. To do that, we have 4 different accounts preconfigured in our core:
1 - no balance available because it’s a new account
2 - 70 USD available
3 - no available balance because everything is either spent or on hold
4 - inactive account
Let’s try sending some money from account 2 to account 1. In reality we would probably process this transfer internally since it is an intrabank transfer, but here for now it will be processed by Ledger.
We should now see both credit and debit requests in the logs like this.
Aborting a request
This may be a good time to show the state diagram for processing eg. credit Entries. Ledger sends prepare
, commit
or abort
requests and we respond with prepared
, failed
, committed
or aborted
statuses, depending on the request and what happened. commit
and abort
requests can not fail and therefore end up in the suspended
state that will have to be fixed internally in the bank. The state diagram for debit Entries looks the same.
State | Description |
---|---|
processing-prepare | This is the first state that an Entry can be in. It is triggered by receiving a prepare request for credit or debit Entries. |
prepared | If a processing-prepare step is successful. We need to notify Ledger of this status. |
failed | If a processing-prepare step fails. We need to notify Ledger of this status. |
processing-commit | If all participants report prepared , Ledger will send commit debit and commit credit requests which will put us into this state. |
committed | If processing-commit step is successful. We need to notify Ledger of this status. |
processing-abort | If Ledger sends an abort request for credit or debit Entries. |
aborted | If processing-abort step is successful. We need to notify Ledger of this status. |
suspended | If processing-commit or processing-abort steps fail. This should not be possible so we have a special status for this. The step is considered successful even if there is an error so it is up to the Bank to fix all steps that end up in this status. |
It is important to note that the abort
request may arrive while we are in processing-prepare
or prepared
states, and will definitely happen if we are in the failed
state after we notify Ledger of the failure. If it happens while we are in prepared
or failed
states, we can process the request (we just need to take into account what state we were in before), but if the abort
request arrives while we are in processing-prepare
, we have set up our code to refuse the request and let Ledger send it again. We could also save this information, respond with 202 Accepted and process it after the current step finishes processing or even better preempt the processing-prepare
step if possible.
Insufficient balance
To begin with, let’s try creating a request that should fail because there is not enough balance in the account.
We see that we get the error we expected: Insufficient available balance in account 3
and after notifying Ledger of the failure POST /v2/debits/:handle/abort was called in the Bridge.
We will now implement this endpoint the same way as the other ones.
We can see that the processing here depends on the state the Entry was already in. If it was prepared
, we need to release the funds, otherwise we don’t have to do anything.
If we now rerun Test 4
, we will get the following logs which is exactly what we expected.
Account inactive
Finally, let’s create an intent that should fail because the target account is inactive.
We see that this time, because the error occurred on the credit side, we got two abort requests.
Let’s implement abort credit the same way we did for debit, we don’t need to do much here.
We can again notice that the only state that we can end up in is aborted
. If something goes wrong, we will end up in the suspended
state and have to fix it internally.
And if we now retry Test 5
we see that it’s fully aborted.
We see that prepare
failed because Account 4 is inactive
so both the credit and debit Entries got the respective abort actions and were aborted.
Sending money to a phone number
Routing money to internal account
Create wallet
Another thing we can try is sending money to a phone number. To do this, we will first set up a phone wallet so that it redirects the money to account:1@mint
.
Create intent
Now let’s try sending some money to this account.
And we should see logs similar to the ones below.
You will notice that, because we have configured the phone wallet to forward the money to account:1@mint
, we now see two intents. Both of them belong to the same Thread.
Aborting account to phone transfer
We will finally do the ultimate test where we redirect money to an inactive internal account. We should see two requests from two different intents and both of them should be aborted.
Create wallet
First we need to create a phone account with the correct route.
Create intent
Next we create a transfer intent to send money to that wallet.
And we get the result we expected! 🍾
Error handling
Until now we have mostly ignored errors or just returned generic values. However, we do need to deal with errors at some point. To begin with, you can check out the Error reference.
For us there will be two main types of errors. Ones that we will respond with directly to all the Entry, Action and Intent requests (which includes HTTP status codes and Ledger error codes) and ones that we will respond with after processing (sending signatures).
There can be many different sources of errors for our application like validation errors, application errors or external errors when communicating with the Core or database (network issues, timeouts and similar). Additionally, when communicating with the Core, we may get not just timeouts, but also HTTP status codes indicating issues or successful responses that themselves indicate an error. You will need to cover all of these cases for your particular situation, but we will give some examples of various errors to serve as a guide.
Conclusion
With everything we have covered so far, we have built a real time payments system between bank accounts. You can now check out the payment initiation tutorial too.