Get started with blockchain using the AWS Hyperledger Fabric template — an unofficial guide (Part 1) DappHero Follow May 23, 2018 · 9 min read

On April 19, 2018, Amazon Web Services made waves by announcing the release of Blockchain Templates for Ethereum and Hyperledger Fabric for its public cloud. At Dapphero, we believe that DLT and blockchain is as powerful as it is complex. We believe templates and best practices are essential for community and industry to adopt and operationalize technologies such as Fabric to unlock its value.

Our curiosity led us to take a look at AWS’s official Hyperledger Fabric template for CloudFormation. While there’s been official coverage from AWS about how to use the Ethereum template, we wanted to understand what the template can do for Hyperledger Fabric users. After a thorough review we wanted to share our journey in a multipart Dapphero blog series to showcase an end-to-end decentralized application on top of Hyperledger Fabric running in AWS.

By the end of this blog series, following the step-by-step instructions, you will be able to accomplish the following:

Leverage AWS’s Hyperledger Fabric template to provision infrastructure for 3 organizations with 1 peer each joining the consortium network. Find your way around the AWS-hosted cloud host plus the basics of the Fabric command line interface. Run multiple instances of IBM’s Marbles decentralized application demo against a consortium network.

This article requires minimal prior knowledge of AWS services, particularly EC2 and CloudFormation.

Marbles is a decentralized application published by IBM under the Apache 2.0 license. This dapp demonstrates transferring marbles between many owners across independent organizations leveraging Hyperledger Fabric as a DLT/blockchain platform. It enforces lifecycle-control, ownership, transfer between parties, and an audit trail of individual marbles’ provenance record. We suggest you check out the project’s extensive readme if you want to learn more about Marbles.

Illustrates three instances, one per org, of IBM’s Marbles dapp running against AWS-hosted Fabric consortium blockchain

Part 1: Leverage AWS’s Hyperledger Fabric template

Let’s jump right into the first section of this multipart blog post series.

Step 1: Fabric infrastructure provisioning using AWS’s CloudFormation template

Navigate to the Getting Started with Blockchain Templates page. Select the link with your desired region under the Hyperledger Fabric section from the right hand side. At Dapphero we will use US West (Oregon), aka us-west-2. If you’re not logged in the AWS console will prompt you to enter your credentials. Fill out the Stack Creation form as illustrated in the screencast below going with the defaults wherever possible. We will also share script snippets that will assume your orgs are called org1, org2, and org3. The name scheme will be important for our scripts to help streamline the provisioning process in subsequent blog posts.

A few things to watch out for:

The CloudFormation template form only applies simple sanity checks, e.g. Security Group and VPC ID that are not associated will go unnoticed and the stack creation will fail late in the process. Be sure to double check ahead of time. For simplicity we chose interact with the AWS host through it’s public network interface. The Security Group (e.g. SG_Fabric_Demo) should allow inbound TCP connections to following ports: 22, 80, 8080, 7050, 7051, 7053, 8051, 8053, 9051, and 9053. You can white list connections from anywhere (PoC only!), specific IP ranges, or use a bastion host (out of scope) to access the cloud machine hosting your org’s peers. We’ve created a temporary IAM role to run the stack creation under. Be sure to grant the role access to provision EC2 infrastructure with the two specific policies, 1.) AmazonEC2ContainerRegistryReadOnly and 2.) AmazonS3ReadOnlyAccess. Needless to say, make sure you’re in possession of the private key in EC2 used to run the template.

Once submitted the CloudFormation will navigate to the stack details page where the status of CREATION_IN_PROGRESS will be displayed. It’ll take a few minutes to complete. Good opportunity to grab a cup of tea/coffee.

CloudFormation template-based stack creation end-to-end

Once completed the stack details will show CREATE_COMPLETE. In case errors occur please note that this CloudFormation template leverages nested stacks (go to the stacks overview page), which in our case, helped diagnose issues taking a closer look at the logging. Please also notice that for demonstration purposes AWS chose to provision a single cloud host instance to host multiple Fabric peers for all orgs. It is not recommended to apply similar setups for production use.

Step 2: Check out Fabric Explorer web app

The AWS template comes with Fabric Explorer (arguably an outdated release) out of the box. Navigating a web browser to following URL will drop right into the Fabric Explorer experience.



(e.g.

http://<Instance IP address>:8080/

(this will also work; e.g. http:// /(e.g. http://54.202.17.239 http:// :8080/(this will also work; e.g. http://54.202.17.239:8080/

Fabric Explorer should resemble the screencast below. A few things to be aware of:

Some browsers no longer autocomplete to http:// when a IP address is entered. They either use https:// which won’t be available or error. We also realized (Thanks, Dave!) that port 8080 will also serve the Explorer as long as the Security Group allows inbound connections. Newly provisioned Explorers might not show much data beyond peers until transactions come in. The AWS template leverages iptables directly to expose the container’s port publicly on port 80. Unfortunately, the iptables instruction won’t persist across machine reboots. Try port 8080 instead. If the Explorer won’t load even though the cloud host wasn’t rebooted, the Security Group might miss instructions to allow inbound TCP connections on port 80. Give port 8080 a shot instead.

Fabric Explorer web interface

As we deploy the Marbles application later, Fabric Explorer is a neat tool to follow along and monitor while transaction data comes into the network.

Step 3: Navigate Fabric through AWS cloud host

The CloudFormation stack details page will display the instance ID in the Output section under DevDesktopInstanceId. Located the Fabric host instance the EC2 console and follow the instructions on how to SSH into the box.

~/demo $ ssh -i yourkey.pem -r ec2-user@<Instance IP address>

> Amazon Linux AMI

The home directory of ec2-user will show two entries that are Hyperledger Fabric related. Both were prepackaged with AWS template.

~ $ ls -l ~/

total 8

drwxr-xr-x 3 ec2-user ec2-user 4096 Apr 24 23:44 HyperLedger-BasicNetwork

drwxr-xr-x 5 ec2-user ec2-user 4096 May 16 17:35 hyperledger-fabric-samples

This is where it helps to realize that AWS chose to deliver Fabric through the Docker container infrastructure using Docker Compose. It’s based off the example projects published by the Hyperledger Fabric project. A glimpse into the right subdirectory reveal the relevant Docker configuration files.

$ ls -l ~/HyperLedger-BasicNetwork/artifacts/docker-compose/

total 40

-rw-r - r - 1 ec2-user ec2-user 4185 May 16 17:36 docker-compose-base.yaml

-rw-r - r - 1 ec2-user ec2-user 4575 Apr 24 23:44 docker-compose-base.yaml.template

-rw-r - r - 1 ec2-user ec2-user 2085 Apr 24 23:44 docker-compose-cli-only.yaml.template

-rw-r - r - 1 ec2-user ec2-user 3662 May 17 00:51 docker-compose-cli.yaml

-rw-r - r - 1 ec2-user ec2-user 3968 Apr 24 23:44 docker-compose-cli.yaml.template

-rw-r - r - 1 ec2-user ec2-user 749 May 16 17:36 docker-compose-explorer.yaml

-rw-r - r - 1 ec2-user ec2-user 790 Apr 24 23:44 docker-compose-explorer.yaml.template

-rw-r - r - 1 ec2-user ec2-user 1006 Apr 24 23:44 peer-base.yaml

Any docker-compose based command will work specifying docker-compose-cli.yaml as respective configuration file. To see what containers are running we can list them and tail all their logs with Docker Compose.

$ cd ~/HyperLedger-BasicNetwork/artifacts/docker-compose/

$ docker-compose -f docker-compose-cli.yaml ps

Name Command State Ports

----------------------------------------------------------------------------------------------------------------

cli /bin/bash Up

fabric-explorer /bin/sh -c /opt/blockchain ... Up 0.0.0.0:8080->8080/tcp

fabric-explorer-db docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp

orderer.example.com orderer Up 0.0.0.0:7050->7050/tcp

peer0.org1.example.com peer node start Up 0.0.0.0:7051->7051/tcp, 0.0.0.0:7053->7053/tcp

peer0.org2.example.com peer node start Up 0.0.0.0:8051->7051/tcp, 0.0.0.0:8053->7053/tcp

peer0.org3.example.com peer node start Up 0.0.0.0:9051->7051/tcp, 0.0.0.0:9053->7053/tcp $ docker-compose -f docker-compose-cli.yaml logs | grep "example.com"

[...]

peer0.org1.example.com | 2018–05–21 22:17:08.307 UTC [gossip/discovery] expireDeadMembers -> WARN 037 Entering [[170 123 98 139 66 14 67 137 217 110 254 192 37 205 91 30 68 48 210 154 215 82 38 145 53 103 71 19 112 148 202 206]]

peer0.org1.example.com | 2018–05–21 22:17:08.307 UTC [gossip/discovery] expireDeadMembers -> WARN 038 Closing connection to Endpoint: peer0.org2.example.com:7051, InternalEndpoint: , PKI-ID: [170 123 98 139 66 14 67 137 217 110 254 192 37 205 91 30 68 48 210 154 215 82 38 145 53 103 71 19 112 148 202 206], Metadata: []

peer0.org1.example.com | 2018–05–21 22:17:08.307 UTC [gossip/discovery] expireDeadMembers -> WARN 039 Exiting

More often than not, being able to get close to the container/process level, makes troubleshooting problems more effective and much quicker. It also offers a great way to learn more about Fabric’s inner workings.

Before we move on let’s take a look where more configuration data relevant to Fabric itself, the Explorer, basic automation scripts, and the underlying crypto material is located on the AWS host.

$ ls -l /etc/fabric/

total 32

drwxr-xr-x 7 root root 4096 May 16 17:36 chaincode

drwxrwxr-x 2 ec2-user ec2-user 4096 May 17 00:51 channel-artifacts

drwxr-xr-x 4 ec2-user ec2-user 4096 May 16 17:36 crypto-config

-rw-r - r - 1 root root 3195 May 16 17:36 explorer-config.json

drwxr-xr-x 2 root root 4096 May 16 17:36 network-management-scripts

drwxr-xr-x 2 root root 4096 May 16 17:36 scripts

drwxr-xr-x 2 root root 4096 May 16 17:36 tools-bin

drwxr-xr-x 2 root root 4096 May 16 17:36 tools-config

Amongst all the artifacts an important data store is the crypto-config directory which stores all organizations certificates, their public and private key pairs. Please note storing this highly sensitive crypto material of all organizations in one single location (for simplicity sake) under control of a single person or entity is not recommended for any form of production-grade operation. The high-level shallow tree of crypto-config should resemble the structure on the directory level below.

$ tree -d -L 3 crypto-config

crypto-config

├── ordererOrganizations

│ └── example.com

│ ├── ca

│ ├── msp

│ ├── orderers

│ ├── tlsca

│ └── users

└── peerOrganizations

├── org1.example.com

│ ├── ca

│ ├── msp

│ ├── peers

│ ├── tlsca

│ └── users

├── [...repeats for org2., org3.]

In a later part of the blog series we will require the crypto-config to run instances of the Marbles application. For now, let’s move on to the Fabric’s command line interface.

Step 4: Explore Fabric peer CLI on AWS host

On a basic operational-level Hyperledger Fabric is a set of few binaries running as processes in containers offering APIs through network interfaces. In order to interact with Fabric one could use an official language specific SDKs or instead use the built-in command line interface to issue calls against the API to perform various Fabric management tasks and monitoring.

Unfortunately, the AWS template shipped with a subtle typo (Dapphero filed a bug report with AWS) that paralyzes the pre-provisioned CLI out of the box. Applying the fix takes a few steps but is overall straightforward.

$ cd HyperLedger-BasicNetwork/artifacts/docker-compose/

$ nano docker-compose-cli.yaml

Remove the blank after /opt in /opt /gopath in line 68 to /opt/gopath and

exit + save with keyboard shortcut CTRL+X followed by confirming the changes. Next up, the CLI container needs to be recreated to pick up the fix followed by launching into a bash inside the cli container.

$ IMAGE_TAG=latest docker-compose -f docker-compose-cli.yaml up -d --no-deps cli

$ docker-compose -f docker-compose-cli.yaml exec cli bash

root@0f45d277b51a:peer# ls

channel-artifacts crypto scripts

Inside the container Fabric’s peer binary offers a variety of commands to introspect what’s going on inside a Fabric blockchain and enact changes. The peer binary picks up on environment variables to figure out what peer to connect to. Following instructions will reveal what peer has joined what channels.

root@0f45d277b51a:peer# export | grep PEER | grep -v TLS

declare -x CORE_PEER_ADDRESS="peer0.org1.example.com:7051"

declare -x CORE_PEER_ID="cli"

declare -x CORE_PEER_LOCALMSPID="org1MSP"

declare -x CORE_PEER_MSPCONFIGPATH="/

root@0f45d277b51a:peer# peer channel list

2018–05–22 15:41:11.235 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized

Channels peers has joined:

mychannel

2018–05–22 15:41:11.237 UTC [main] main -> INFO 002 Exiting….. declare -x CORE_PEER_ADDRESS="peer0.org1.example.com:7051"declare -x CORE_PEER_ID="cli"declare -x CORE_PEER_LOCALMSPID="org1MSP"declare -x CORE_PEER_MSPCONFIGPATH="/ opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com /msp"2018–05–22 15:41:11.235 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initializedChannels peers has joined:mychannel2018–05–22 15:41:11.237 UTC [main] main -> INFO 002 Exiting…..

To issue commands against org2’s or org3’s peers it’s a matter of setting the environment to match address, MSP, and MSPIDs of the respective orgs. The peer binary offers help under peer channel help and the Peer Command Reference part of the online documentation.

What’s next?

Congrats! Now that we understand the set up of the AWS host provisioned through the official AWS template in the next article we will focus on how to navigate identity without Fabric Certificate Authorities (not provisioned in AWS template), deploy chaincodes (Fabric-speak for smart contracts), and run multiple instances of Marbles to showcase how decentralized applications work end-to-end in a consortium network.

We are looking forward to your feedback and questions in the comments below or through Dapphero’s contact form.