Alright so I tried to transplant the hard drive from my initial controller node into a new system but had no luck. I tried a few things to fix the system so it would boot but they didn’t work. So now I’m installing fresh on the new system.

This time around it seems that I may be encountering new problems. Right off the bat I’m having an issue starting the httpd service. Trying to start it generates a failure message and checking the log show that permission to bind to all addresses on port 5000 for keystone is being denied. Interestingly enough this problem is being caused by SELinux. A few minutes searching google has led me to this page which had the information I needed. Quickly disabling SELinux allowed me to start httpd and confirmed that it was the issue. Following the links at the bottom of the previous page brings me to this page which tells me how to configure SELinux to permanently work in permissive mode.

A lot of my other errors seem to be typos though. I’m trying to rush through this and it seems that’s a bad approach when dealing with many config files.

horizon

I encountered more issues after getting the dashboard installed as well. The install went fine, but when I went to log in it wasn’t letting me log in with either the admin or the demo credentials. Like most of the issues I’ve had google searches turn up results that aren’t relevant. I looked through all the logs and couldn’t find any obvious errors either. I knew it had to be a configuration issue but it took me quite a while to find it.

The issue was that the default host endpoint in the configuration file was wrong for this install and it wasn’t a really obvious mismatch either. All the configurations for horizon take place in /etc/openstack-dashboard/local_settings. In particular the problem child was OPENSTACK_KEYSTONE_URL. It was by default set to http://%s:5000/v3.0 when my system is configured to use /v3. Not really obvious, especially when you’ve been doing a lot of configurations of things that can’t tell the difference between 3.0 and 3. That change and restart the httpd service allowed me to log in to horizon.

I’m finally at the same point I was at before moving controllers. Actually as it turns out I’m better off than I was. After setting up the image, the flavor, and a basic network I found that I was actually able to spin up instances! My theory about the under powered system was right! I’m even noticing that the web UI is much more responsive this time around and certain commands on the controller node are running faster as well.

However I wasn’t out of the water yet. I could spin up a VM, but could I get a console connection? Unfortunately the answer was no. Again google results were less than helpful. Like most of my issues have been though I was suspicious of the firewall. Disabling the firewall and restarting showed no change so my next step was selinux. A quick “setenforce 0” and a restart of nova proved it was in fact selinux. I have a console! This means that my openstack deployment is fully functional! A quick login to each compute node and I set the option “Enforcing” to “disabled” in /etc/selinux/config and now my compute hosts can restart without having to worry about consoles not connecting.

I’m very happy to have a working stack now. Looking back there are a lot of things I’ll do during deployment on my next trip down deployment lane. Instead of disabling firewalls and selinux I’d like to actually just configure the proper ports for security reasons. That should make things a lot smoother. There are a few key points where a restart is helpful as well.

Interesting addition here. After playing around for a while I finally got one VM running on each of my compute nodes (couldn’t do live migrations without ephemeral storage) and one of my nodes isn’t serving VNC properly. It’s pointing me to 127.0.0.1 and my desktop is refusing to connect for some reason. It’s like it’s not even a member of this cluster! Oh wait…. So now I have to trouble shoot why one compute node is working with vnc and the other is not. Edit: Found the culprit. It was another typo in a config file. The compute host was defaulting to a vnc proxy url of 127.0.0.1 because I had one too many p’s in “novncproxy”. All is well now.

I’m planning on doing a post to reflect completely on the challenges I faced and the things I learned from the experience. Overall I’m satisfied that I was able to conquer at least a very basic OpenStack deployment.

Hopefully someone will find these posts helpful for deploying their own OpenStack! I’ll be sure to continue to post as I start to add more services like block and object storage as I’m sure I’ll face plenty of challenges with those as well.

Catch you next time!