I explained both BOSH and the Photon platform in previous posts. I never did a post on how to deploy BOSH on vSphere but this document does a very good job describing the process. The only thing I want to add to that is: Don’t use “@” in your passwords! Cost me a day or so to figure out what was going wrong. In this post I will detail how to run BOSH on VMware Photon platform.

Update 19-04-2017: This post was based on Photon platform 1.1.1. As of today the current version is Photon platform 1.2. The steps in this post may or may not work for version 1.2.

Prepare Photon Platform

Install Photon platform. This blog post details how you to do that. Make sure you have the photon cli installed. Instructions here. I’m going to assume that you don’t have anything configured on the photon platform yet. If you have you’ll probably already know what to do. I’ll also ussume this is a lab where you have full access. Connect the photon cli to you photon platform.

photon target set https://192.168.192.76:443 --nocertcheck photon target login --username administrator@photon.lab 1 2 photon target set https : / / 192.168.192.76 : 443 -- nocertcheck photon target login -- username administrator @ photon .lab Create a photon tenant and tell the cli to use it (press enter on any questions to use the default)

photon tenant create bosh photon tenant set bosh 1 2 photon tenant create bosh photon tenant set bosh Create a network. I’m going to assume you use the default portgroup named “VM Network”. If not please substitute your network name in the command below.

photon network create --name vmnet --portgroups 'VM Network' --description vmnet 1 photon network create -- name vmnet -- portgroups 'VM Network' -- description vmnet Create a resource ticket for the bosh environment. I didn’t find a way to deploy to other projects than the one you deployed the bosh director to. So make sure you create a big enough ticket to also fit the workloads you’ll be deploying with BOSH.

photon resource-ticket create --name boshTicket --tenant bosh --limits 'vm.count 100 COUNT, vm.memory 32 GB, vm.cpu 400 COUNT' 1 photon resource - ticket create -- name boshTicket -- tenant bosh -- limits 'vm.count 100 COUNT, vm.memory 32 GB, vm.cpu 400 COUNT' Create a project that consumes the resources.

photon project create --name boshProject --tenant bosh --resource-ticket boshTicket --percent 100 1 photon project create -- name boshProject -- tenant bosh -- resource - ticket boshTicket -- percent 100 Add some flavor. Flavors are types of resources on offer on the Photon platform. It’s like AWS instance types.

photon flavor create --name vm-basic --kind vm --cost 'vm.count 1 COUNT, vm.cpu 2 COUNT, vm.memory 2 GB' photon flavor create --name disk-eph --kind ephemeral-disk --cost 'ephemeral-disk.count 1 COUNT' photon flavor create --name disk-persist --kind persistent-disk --cost 'persistent-disk.count 1 COUNT' 1 2 3 photon flavor create -- name vm - basic -- kind vm -- cost 'vm.count 1 COUNT, vm.cpu 2 COUNT, vm.memory 2 GB' photon flavor create -- name disk - eph -- kind ephemeral - disk -- cost 'ephemeral-disk.count 1 COUNT' photon flavor create -- name disk - persist -- kind persistent - disk -- cost 'persistent-disk.count 1 COUNT'

Deploy BOSH

Install BOSH cli tools

To be able to install BOSH you’ll need the bosh-init tool. This tool is like a mini BOSH which is able to deploy BOSH. So it’s kinda like BOSH deploys itself. I won’t explain how to install bosh-init, the cloud foundry docs on this are pretty good. Please follow instructions here.

To be able to interact with a BOSH director once it’s deployed you’ll need the BOSH cli itself. In this case you’ll even need it before the BOSH director is running because it’s used to build the Photon CPI release. Again, find the cloud foundry docs on how to install the bosh cli here.

Prepare the Photon CPI

BOSH is able to work with a lot of different cloud (IaaS) providers and platforms. I already mentioned vSphere. But BOSH is also able to use vCloud, AWS, Google and Openstack. The magic that makes this multi-cloud solution possible is the Cloud Provider Interface or CPI.

VMware has published a CPI for Photon. It’s not published on the BOSH website yet but you can find it on github. To be able to use the CPI you’ll want to install it into a BOSH director. How? Using a BOSH release of course. The Photon CPI BOSH release is here. Since there is no ready build Photon CPI release we’ll have to build our own. Don’t be scared, it’s not that hard (disclaimer: I’m using Ubuntu. commands on a Mac should be similar, not sure about window though). Here we go:

Make sure you have the git client installed on your OS Create a folder to contain the CPI release and your deployment yml. I used ~/my-bosh/photon. cd into the folder you created Clone the Photon CPI release git repo, cd into the created folder and create the release:

git clone https://github.com/cloudfoundry-incubator/bosh-photon-cpi-release.git cd ./bosh-photon-cpi-release/ bosh create release --force --with-tarball 1 2 3 git clone https : / / github .com / cloudfoundry - incubator / bosh - photon - cpi - release .git cd . / bosh - photon - cpi - release / bosh create release -- force -- with - tarball There’ll be a dev_releases folder in the bosh-photon-cpi-release folder now. Copy the cpi tgz file to ~/my-bosh/photon

cd ~/my-bosh/photon cp ./bosh-photon-cpi-release/dev_releases/bosh-photon-cpi/bosh-photon-cpi-1.1.1+dev.1.tgz ./ 1 2 cd ~ / my - bosh / photon cp . / bosh - photon - cpi - release / dev_releases / bosh - photon - cpi / bosh - photon - cpi - 1.1.1 + dev . 1.tgz . /

Create BOSH manifest

deployments in BOSH are described in so called manifests. These are files in YAML format containing a bunch of settings. Each type of deployment has it’s own manifest and so does the bosh deployment itself.

You can find an example manifest for bosh with the photon CPI in the photon CPI release git repo. I’ll share my own manifest below so you ‘ll have a feel of what it should look like with all the values populated. If you used the yml from my blog post to deploy photon then you can use the my bosh manifest with just two changes:

change the network id on line 39. The command to get the id is

photon network list 1 photon network list Change the photon project id on line 114. The command to get the id is

photon project list 1 photon project list

save the manifest yml to ~/my-bosh/photon/bosh-photon.yml

--- name: bosh releases: - name: bosh url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=261.4 sha1: 4da9cedbcc8fbf11378ef439fb89de08300ad091 - name: bosh-photon-cpi url: file://bosh-photon-cpi-1.1.1+dev.1.tgz resource_pools: - name: vms network: private stemcell: url: https://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent?v=3363.12 sha1: 8899d9b76edde5722d98088983d416fa32c597e9 cloud_properties: vm_flavor: vm-basic disk_flavor: disk-eph env: bosh: # c1oudc0w is a default password for vcap user password: "$6$4gDD3aV0rdqlrKC$2axHCxGKIObs6tAmMTqYCspcdvQXh3JJcvWOY2WGb4SrdXtnCyNaWlrf3WEqvYR2MYizEGp3kMmbpwBC6jsHt0" disk_pools: - name: disks disk_size: 20_000 cloud_properties: disk_flavor: disk-persist networks: - name: private type: manual subnets: - range: 192.168.192.0/24 gateway: 192.168.192.1 dns: [192.168.192.21] cloud_properties: network_id: 53bf297f-bfd7-45f3-9701-5e86448baefd jobs: - name: bosh instances: 1 templates: - {name: nats, release: bosh} - {name: postgres, release: bosh} - {name: blobstore, release: bosh} - {name: director, release: bosh} - {name: health_monitor, release: bosh} - {name: cpi, release: bosh-photon-cpi} resource_pool: vms persistent_disk_pool: disks networks: - {name: private, static_ips: [192.168.192.40]} properties: nats: address: 127.0.0.1 user: nats password: password postgres: &db listen_address: 127.0.0.1 host: 127.0.0.1 user: postgres password: password database: bosh adapter: postgres blobstore: address: 192.168.192.40 port: 25250 provider: dav director: user: director password: password agent: user: agent password: password options: endpoint: http://192.168.192.40:25250 user: agent password: password director: address: 127.0.0.1 name: my-bosh db: *db cpi_job: cpi user_management: provider: local local: users: - {name: admin, password: password} - {name: hm, password: password} hm: director_account: user: hm password: password resurrector_enabled: true intervals: agent_timeout: 180 photon: &photon target: https://192.168.192.76:443 user: administrator@photon.lab password: Passw0rd123! ignore_cert: true project: 9c6103d7-b7e1-4850-bd17-bad87570be3f agent: mbus: nats://nats:password@192.168.192.40:4222 ntp: &ntp [nl.pool.ntp.org] cloud_provider: template: {name: cpi, release: bosh-photon-cpi} mbus: https://mbus:password@192.168.192.40:6868 properties: photon: *photon agent: {mbus: 'https://mbus:password@0.0.0.0:6868'} blobstore: provider: local options: blobstore_path: /var/vcap/micro_bosh/data/cache ntp: *ntp 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 --- name : bosh releases : - name : bosh url : https ://bosh.io/d/github.com/cloudfoundry/bosh?v=261.4 sha1 : 4da9cedbcc8fbf11378ef439fb89de08300ad091 - name : bosh-photon-cpi url : file ://bosh-photon-cpi-1.1.1+dev.1.tgz resource_pools : - name : vms network : private stemcell : url : https ://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent?v=3363.12 sha1 : 8899d9b76edde5722d98088983d416fa32c597e9 cloud_properties : vm_flavor : vm-basic disk_flavor : disk-eph env : bosh : # c1oudc0w is a default password for vcap user password : "$6$4gDD3aV0rdqlrKC$2axHCxGKIObs6tAmMTqYCspcdvQXh3JJcvWOY2WGb4SrdXtnCyNaWlrf3WEqvYR2MYizEGp3kMmbpwBC6jsHt0" disk_pools : - name : disks disk_size : 20_000 cloud_properties : disk_flavor : disk-persist networks : - name : private type : manual subnets : - range : 192.168.192.0/24 gateway : 192.168.192.1 dns : [192.168.192.21] cloud_properties : network_id : 53bf297f-bfd7-45f3-9701-5e86448baefd jobs : - name : bosh instances : 1 templates : - { name : nats , release : bosh } - { name : postgres , release : bosh } - { name : blobstore , release : bosh } - { name : director , release : bosh } - { name : health_monitor , release : bosh } - { name : cpi , release : bosh-photon-cpi } resource_pool : vms persistent_disk_pool : disks networks : - { name : private , static_ips : [192.168.192.40] } properties : nats : address : 127.0.0.1 user : nats password : password postgres : &db listen_address : 127.0.0.1 host : 127.0.0.1 user : postgres password : password database : bosh adapter : postgres blobstore : address : 192.168.192.40 port : 25250 provider : dav director : user : director password : password agent : user : agent password : password options : endpoint : http ://192.168.192.40 :25250 user : agent password : password director : address : 127.0.0.1 name : my-bosh db : *db cpi_job : cpi user_management : provider : local local : users : - { name : admin , password : password } - { name : hm , password : password } hm : director_account : user : hm password : password resurrector_enabled : true intervals : agent_timeout : 180 photon : &photon target : https ://192.168.192.76 :443 user : administrator@photon.lab password : Passw0rd123! ignore_cert : true project : 9c6103d7-b7e1-4850-bd17-bad87570be3f agent : mbus : nats ://nats :password@192.168.192.40 :4222 ntp : &ntp [nl.pool.ntp.org] cloud_provider : template : { name : cpi , release : bosh-photon-cpi } mbus : https ://mbus :password@192.168.192.40 :6868 properties : photon : *photon agent : { mbus : 'https ://mbus :password@0.0.0.0 :6868' } blobstore : provider : local options : blobstore_path : /var/vcap/micro_bosh/data/cache ntp : *ntp

Run bosh-init deploy

Now you can finally start the deployment. It’s very simple 🙂

cd ~/my-bosh/photon bosh-init deploy ./bosh-photon.yml 1 2 cd ~ / my - bosh / photon bosh - init deploy . / bosh - photon . yml

And now we wait 🙂

Use BOSH

Now that we deployed BOSH we might want to try to use BOSH for something useful. One of the simplest examples of something useful is deploying a redis server. Here are the steps involved:

On the Photon platform create another resource ticket and a new project consuming the ticket. Target the bosh cli to the fresh BOSH director and login (if you’re using my yml the password is ‘password’

bosh target 192.168.192.40 bosh login admin 1 2 bosh target 192.168.192.40 bosh login admin run bosh status to confirm you’re connected and BOSH is up and running. Upload the ubuntu trusty stemcell

bosh upload stemcell https://s3.amazonaws.com/bosh-core-stemcells/vsphere/bosh-stemcell-3363.14-vsphere-esxi-ubuntu-trusty-go_agent.tgz 1 bosh upload stemcell https : / / s3 .amazonaws .com / bosh - core - stemcells / vsphere / bosh - stemcell - 3363.14 - vsphere - esxi - ubuntu - trusty - go_agent .tgz Upload the redis release

bosh upload release http://bosh.io/d/github.com/cloudfoundry-community/redis-boshrelease?v=12 1 bosh upload release http : / / bosh .io / d / github .com / cloudfoundry - community / redis - boshrelease ? v = 12 Create a cloud-config YAML for BOSH. Below is my yml. Replace the project id on line 17 Configure you ip range in lines 37..41 Replace the network id in line 42

azs: - name: homelab vm_types: - name: default cloud_properties: vm_flavor: vm-basic disk_flavor: disk-eph bosh: password: $6$Y2VgK5cR75B1G$CecLGT4euQoKHs9hI1GBk9GXONr1ntlPHFdlWrtkXgEC4bh/1WX.HBT.ygR0WVWq.aV8Vg/cuTi1JbRNXmNJy1 disk_types: - name: default disk_size: 3_000 cloud_properties: disk_flavor: disk-persist - name: medium disk_size: 20_000 cloud_properties: disk_flavor: disk-persist - name: large disk_size: 50_000 cloud_properties: disk_flavor: disk-persist networks: - name: default type: manual subnets: - range: 192.168.192.0/24 gateway: 192.168.192.1 az: homelab dns: [192.168.192.21] reserved: [ 192.168.192.2-192.168.192.150 ] cloud_properties: {network_id: 53bf297f-bfd7-45f3-9701-5e86448baefd} compilation: workers: 4 reuse_compilation_vms: true az: homelab vm_type: default network: default env: bosh: password: $6$hVE/3gPUys$BIgRmQXrAy6lh3YVwjTYqWueqYUhhw7UQkOUFzf3p44bMmZnp5XDvpDebkGjR1om3ot.1jCNYNiXLggLI8Dw1. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 azs : - name : homelab vm_types : - name : default cloud_properties : vm_flavor : vm-basic disk_flavor : disk-eph bosh : password : $6$Y2VgK5cR75B1G$CecLGT4euQoKHs9hI1GBk9GXONr1ntlPHFdlWrtkXgEC4bh/1WX.HBT.ygR0WVWq.aV8Vg/cuTi1JbRNXmNJy1 disk_types : - name : default disk_size : 3_000 cloud_properties : disk_flavor : disk-persist - name : medium disk_size : 20_000 cloud_properties : disk_flavor : disk-persist - name : large disk_size : 50_000 cloud_properties : disk_flavor : disk-persist networks : - name : default type : manual subnets : - range : 192.168.192.0/24 gateway : 192.168.192.1 az : homelab dns : [192.168.192.21] reserved : [ 192.168.192.2-192.168.192.150 ] cloud_properties : { network_id : 53bf297f-bfd7-45f3-9701-5e86448baefd } compilation : workers : 4 reuse_compilation_vms : true az : homelab vm_type : default network : default env : bosh : password : $6$hVE/3gPUys$BIgRmQXrAy6lh3YVwjTYqWueqYUhhw7UQkOUFzf3p44bMmZnp5XDvpDebkGjR1om3ot.1jCNYNiXLggLI8Dw1. Load the cloud config into bosh

bosh update cloud-config ~/my-bosh/photon/photon-cloud-config.yml 1 bosh update cloud - config ~ / my - bosh / photon / photon - cloud - config .yml Create the redis deployment yaml. Again, below is my version of it. Replace the director_uuid. Retrieve the uuid by running bosh status Store the manifest in ~/my-bosh/photon/redis.yml

name: redisOnPhoton director_uuid: c1fe3423-9b10-42e0-95f9-e06053ec38ee releases: - name: redis version: latest instance_groups: - name: redis-master instances: 1 azs: [ homelab ] jobs: - name: redis release: redis properties: redis: password: password vm_type: default stemcell: default persistent_disk_type: medium networks: - name: default update: canaries: 1 canary_watch_time: 1000-100000 max_in_flight: 1 update_watch_time: 1000-100000 stemcells: - alias: default os: ubuntu-trusty version: latest 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 name : redisOnPhoton director_uuid : c1fe3423-9b10-42e0-95f9-e06053ec38ee releases : - name : redis version : latest instance_groups : - name : redis-master instances : 1 azs : [ homelab ] jobs : - name : redis release : redis properties : redis : password : password vm_type : default stemcell : default persistent_disk_type : medium networks : - name : default update : canaries : 1 canary_watch_time : 1000-100000 max_in_flight : 1 update_watch_time : 1000-100000 stemcells : - alias : default os : ubuntu-trusty version : latest Tell the bosh cli to use this manifest

bosh deployment ~/my-bosh/photon/redis.yml 1 bosh deployment ~ / my - bosh / photon / redis .yml Now deploy redis

bosh deploy 1 bosh deploy

After the deployment is finished you can list the deployments and the VMs it deployed but running these commands

bosh deployments bosh vms 1 2 bosh deployments bosh vms

The output should be similar to this:

Pfew….. if you made it this far: Congrats! you’re on your way to being a cloud native :).