🤖 Service Account

Before you go ahead with submitting a new request for your IT infrastructure change, make sure you have a service account as well.

Unlike normal user accounts, service accounts are not tied to a particular employee. This is very important. When a person leaves a company, their account eventually gets wiped and CI pipelines get broken, so the lesson is: never use personal accounts for CI setup. Service accounts don’t ask for a pay raise, don’t chuck a sickie, and they just don’t quit.

Another good thing about service accounts—they normally don’t have the password expiry policy set. You don’t have to update your CI configuration with a new password every 90 days or so.

If you don’t have a service account, request one then. Yes, it may mean you have to open yet another request to some other DevOps team, but who said enterprise was easy?

Using Fastlane behind the Proxy

Fast-forward N weeks, where N is a random large non-negative number, and you have the following at your disposal:

A service account username and password, and maybe even an email, if you’re super lucky.

Let’s assume the username is part of the Active Directory Domain and comes in a au\username form, where au is the domain.

form, where is the domain. Let’s also assume the password is just password .

. All requested domains are whitelisted on all levels of intranet.

You service account can authenticate to the company proxy using username and password and access whitelisted domains.

Let’s assume the proxy host name is company-proxy and it listens on port 8080 for all incoming connections.

Curl Test

First off, run a simple curl test to make sure everything works:

The --proxy-user and --proxy options are self-explanatory, though note the backslash escaping as \\ .

By saying --proxy-anyauth we tell curl to work with any authentication scheme that the proxy supports.

Most likely, all outgoing network traffic on your dev machine is signed with your company’s own self-signed Root Certificate Authority (CA) certificate, so we use the --insecure option to convince curl to accept this self-signed certificate.

The self-signed root CA and authentication scheme will play very important roles further on.

The output of the command should be something like this:

Unauthenticated Request ID: ABCABCBACBACXYZXYZXYZZ37.0.0

This is good. It means the request went all the way to appstoreconnect.apple.com.

🔏 Root CA and SSL Certificates

Just like with curl , a self-signed root CA certificate won’t be accepted by Fastlane either. While running Fastlane, you won't have any option like the curl 's --insecure flag. Instead you need to add your internal root CA certificate to rubygems path, because Fastlane is a Ruby gem:

You may have to use sudo if you’re using the system Ruby version, though I'd recommend using RVM or rbenv to manage your rubies.

If you are using RVM, you’ll need to run the following as well:

Now you can use this Ruby script to test the SSL connection.

E.g. just run it directly like this:

💲 Environment Variables

The next thing you’ll notice is that Fastlane doesn’t have any option for passing proxy information. Luckily, Fastlane is being a good citizen and respects all the /http[s]_proxy/i environment variables, namely:

HTTP_PROXY

HTTPS_PROXY

http_proxy

https_proxy

Remember this “Gang of Four”—nothing else will cause you as much pain as this bunch!

Somewhere in your ~/.bash_profile or other appropriate shell profile, set these environment variables—for example:

Note that proxy URL is set in the following format:

http:// <domain> \ <username> : <password> @ host : port

Without the username and password, you wouldn’t be able to authenticate with your proxy.

Now give it a try. Run the simple download_screenshots deliver action:

Answer all the prompts and see if it worked. I bet it didn’t!

Long story short, Fastlane is using the faraday HTTP client Ruby library, which uses Kernel.URI method, which can’t handle a backslash \ in the username.

The solution is to use percent-encoding for \ , i.e. %5c :

You should be able to download screenshots now, though that’s just a first, tiny step towards the final goal. Well, actually, you can do a lot of things now, such as registering devices, managing App IDs and provisioning profiles, and more. But I’d assume you also want to upload a new ipa to TestFlight, so meet The Transporter then...

🚎 Transporter

Transporter, aka iTMSTransporter, is a Java-based command-line tool used to “upload things to the App Store”, including app metadata and new ipa builds.

Installing Transporter

So how do you get a copy?

Well…according to Apple, you first login to your developer account and then click the download link like this one.

But wait! You can’t download it unless you have an Admin or Technical role.

I mean, “Why, Apple?! Why?!” 😱

Transporter is available as part of the Xcode installation anyways…why make things so hard?

Why not make it available at Apple Developers Downloads at the very least?

Anyways, if you choose to use the copy bundled with Xcode, then you can find it at:

/Applications/Xcode.app/Contents/Applications/Application Loader.app/Contents/itms

Just grab the whole directory.

If you installed it via the Mac OS package, then get it at:

/usr/local/itms

Patching Transporter

If you’re wondering: “Why do I need a copy of Transporter? Can’t Fastlane just use it as is?”

Good question. You don’t need a copy of Transporter. You can use the one available inside Xcode or in your /usr/local , but you would have to patch it before using it anyways.

Given that Transporter is a Java-based command-line app, it has the same issue as curl or any Ruby gem would have with your company's internal self-signed root CA certificate - it won't trust it. Apple didn't bother to add a command line or configuration option of any kind to Transporter to trust self-signed certificates.

Thankfully, Transporter isn’t just a Java-based app, it comes with its own version of Java runtime bundled up inside itms/java . Try running itms/java/bin/java -version to see more info.

Now, the itms/java folder also contains lib/security/cacerts file, which is a keystore with CA certificates. So you need to add your company certificate to cacerts , and itms/java/bin/keytool is just the right tool to do that.

Here ${ROOT_CA_PATH} is the path to the certificate file, which you can locate directly or find in the OS X keychain

Now you’ve convinced Transporter to trust your root CA, but you’re not there yet, because you also need to teach Transporter some respect towards your new and shiny proxy, which is where trouble awaits for you…again.

Transporter and Proxy

Now that we know that Transporter is a Java-based app, we look closer inside the itms folder and find itms/java/lib/net.properties - a Java properties file used to configure the JVM that Transporter runs in.

The group of proxy variables is exactly what we need:

You don’t even have to modify the net.properties file. Fastlane reads the value of the special DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS environment variable and uses it to configure Transporter. For example, in your bash script add the following export statement:

But will it work? By now you’ve been there enough times to know the answer — “No”. 😜

Proxy configuration needs to include the username and password for Basic authentication.

You can try to add au%5cusername:password@company-proxy as the host value, but it won’t work.

Removing the jdk.http.auth.tunneling.disabledSchemes=Basic limitation from net.properties won't help either.

While trying to Google you may find mention of these properties:

Those are part of Apache HTTP Client JVM options, though, and are not part of Oracle’s default network properties. Also, Transporter doesn’t respect them.

🚧 Feels like another roadblock.

Another Proxy

The solution this time is to create yet another proxy (as if there weren’t enough problems with them already!).

This “man in the middle” local proxy will take care of passing authentication information to your company proxy. The credentials no longer have to be hard-coded in a proxy URL. This way, the command line tools like Transporter can use a local proxy for network connections, and a local proxy doesn’t require any type of authentication.

There are a number of proxy tools available for Mac OS. In this example, I’ll be using CNTLM.

I’ll briefly mention that I also tried TinyProxy and ProxyChains-NG. TinyProxy just wouldn’t let me use \ or %5c in the configuration file, while with ProxyChains-NG I eventually just gave up after a number of failed attempts to make it work.

So, CNTLM—a fast proxy for NTLM authentication implemented in C.

Easy to install.

Also easy to configure. Just put the following in cntlm.conf :

Now run this command to generate password hashes:

Type in your account password, copy hashes, and append them to cntlm.conf :

Then run it:

Finally, configure proxy environment variables:

Give it a go!

What do you mean “It doesn’t work!?” Something about the proxy returning an invalid challenge?

Well, that just means you company proxy doesn’t support NTLM authentication and only supports Basic authentication. That needs to be changed by requesting IT support again, I’m afraid.