This post is a log of the work I am doing for bug 905742 so that I keep track of activities.

Firstly, I have grabbed a slave from slavealloc: bld-lion-r5-087.build.releng.scl3.mozilla.com.

There is a useful developers guide which I have been reading: Preparing for your first B2G build

The first problem is that I cannot follow these steps directly, since they require that you have access to git://github.com - and we don’t from the network where our slaves reside. Also, we don’t *want* to get from external repos, we want to use our own (less network latency and we are in control of availability).

Therefore, I have started work on the following script, taking steps from this recent b2g_mozilla-central_emulator_dep build that already exists (for linux builders) with the intention to find out what (if anything) needs to be adapted to get it to run successfully on OS X. I will also compare this to a recent b2g_mozilla-central_emulator-jb_dep build to see what the differences are when building against Jelly Bean (the first builder is for Ice Cream Sandwich).

Work in progress:

#!/bin/bash -exv cd "$(dirname "${0}")"



### need this variable for config, see: https://hg.mozilla.org/build/mozharness/file/8ca3f8e6bce9/scripts/b2g_build.py#l172

export PROPERTIES_FILE="$(pwd)/buildprops.json"

rm -rf build-dir mkdir -p build-dir cd build-dir

hg clone 'http://hg.mozilla.org/build/mozharness' scripts

cd scripts

hg update -C -r production

hg id -i

cd ..

'scripts/scripts/b2g_build.py' '--target' 'generic' '--config' 'b2g/releng-emulator.py' '--b2g-config-dir' 'emulator' '--gaia-languages-file' 'locales/languages_dev.json' '--gecko-languages-file' 'gecko/b2g/locales/all-locales'

cd properties

for file in "$(ls -1)"

do

cat "${file}"

done

In order for this to run, a buildprops.json file is needed. When building under buildbot, this file is passed to the slave from the master by a call to this method during the download_props build step. The properties that are retrieved corresponds to the “Build Properties” that are displayed in this page of the buildbot web interface.

Since my example linux builder is currently down, I have grabbed a different buildprops.json from a different, unrelated slave, to make sure I have the format approximately correct, have reformatted the file to make it more readable, and am currently editing the contents to be suitable.

Here is the buildprops.json file I am currently using:

{ "properties": { "base_bundle_urls": "['http://ftp.mozilla.org/pub/mozilla.org/firefox/bundles']", "base_mirror_urls": "['http://hg-internal.dmz.scl3.mozilla.com']", "basedir": "/Users/cltbld/pmoore/build-dir", "branch": "mozilla-central", "buildername": "b2g_mozilla-central_emulator_dep", "buildid": "20131112184405", "buildnumber": 97, "builduid": "a6610df53d7b442aba89c773e8e8c104", "compare_locales_revision": "1fc4e9bc8287", "gaia_revision": "9abc79e45a7283a557f177a3f9b54bec2668a2b8", "gecko_revision": "7b014f0f3b03", "hgurl": "http://hg.mozilla.org/", "master": "http://buildbot-master64.srv.releng.usw2.mozilla.com:8001/", "mock_target": "None", "platform": "emulator", "product": "b2g", "project": "", "repo_path": "mozilla-central", "repository": "", "revision": "7b014f0f3b03", "scheduler": "b2g_mozilla-central-b2g", "script_repo_revision": "7034f553e9cd", "slavename": "bld-lion-r5-087", "upload_ssh_key": "ffxbld_dsa", "upload_ssh_server": "stage.mozilla.org", "upload_ssh_user": "ffxbld" }, "sourcestamp": { "branch": null, "changes": [], "hasPatch": false, "project": "", "repository": "", "revision": null } }

So far I have had to make the following adjustments…

1. Special case of where b2g_manifest_branch = ‘master’

One of the mozharness scripts (b2g_build.py) initialises a local empty git repository (tmp_manifest), adds and commits a file (a manifest file called generic.xml), and then renames the active branch of the working directory to the required value (see code here). However, in the special case that the target branch name (defined in variable b2g_manifest_branch) is 'master’, this will try to rename the active branch from its existing branch name (also 'master’), to this new name ('master’). On our linux machines this seems not to be a problem (maybe the version of git is smart enough to realise no rename is necessary if source and target names are the same), but on the darwin build slave I am using, the git branch command fails with “fatal: A branch named 'master’ already exists.”

To get around this, we can use -M instead of -m in the branch command, which allows you to replace an existing branch name.

Example linux output (taken from here):

18:44:43 INFO - Running command: ['git', 'init'] in /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest 18:44:43 INFO - Copy/paste: git init 18:44:43 INFO - Initialized empty Git repository in /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest/.git/ 18:44:43 INFO - Return code: 0 18:44:43 INFO - Running command: ['git', 'add', '/builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest/generic.xml'] in /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest 18:44:43 INFO - Copy/paste: git add /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest/generic.xml 18:44:43 INFO - Return code: 0 18:44:43 INFO - Running command: ['git', 'commit', '-m', 'manifest'] in /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest 18:44:43 INFO - Copy/paste: git commit -m manifest 18:44:43 INFO - [master (root-commit) 3803f02] manifest 18:44:43 INFO - 1 file changed, 106 insertions(+) 18:44:43 INFO - create mode 100644 generic.xml 18:44:43 INFO - Return code: 0 18:44:43 INFO - Running command: ['git', 'branch', '-m', u'master'] in /builds/slave/b2g_m-cen_emu_dep-000000000000/build/tmp_manifest 18:44:43 INFO - Copy/paste: git branch -m master 18:44:43 INFO - Return code: 0

Equivalent example darwin output (copied from the session on my test run):

09:41:56 INFO - Running command: ['git', 'init'] in /Users/cltbld/pmoore/build-dir/build/tmp_manifest

09:41:56 INFO - Copy/paste: git init

09:41:56 INFO - Initialized empty Git repository in /Users/cltbld/pmoore/build-dir/build/tmp_manifest/.git/

09:41:56 INFO - Return code: 0

09:41:56 INFO - Running command: ['git', 'add', '/Users/cltbld/pmoore/build-dir/build/tmp_manifest/generic.xml'] in /Users/cltbld/pmoore/build-dir/build/tmp_manifest

09:41:56 INFO - Copy/paste: git add /Users/cltbld/pmoore/build-dir/build/tmp_manifest/generic.xml

09:41:56 INFO - Return code: 0

09:41:56 INFO - Running command: ['git', 'commit', '-m', 'manifest'] in /Users/cltbld/pmoore/build-dir/build/tmp_manifest

09:41:56 INFO - Copy/paste: git commit -m manifest

09:41:56 INFO - [master (root-commit) 5f63ccb] manifest

09:41:56 INFO - 1 files changed, 106 insertions(+), 0 deletions(-)

09:41:56 INFO - create mode 100644 generic.xml

09:41:56 INFO - Return code: 0

09:41:56 INFO - Running command: ['git', 'branch', '-m', u'master'] in /Users/cltbld/pmoore/build-dir/build/tmp_manifest

09:41:56 INFO - Copy/paste: git branch -m master

09:41:56 INFO - fatal: A branch named 'master' already exists.

09:41:56 ERROR - Return code: 128

09:41:56 FATAL - Halting on failure while running ['git', 'branch', '-m', u'master']

09:41:56 FATAL - Running post_fatal callback...

09:41:56 FATAL - Exiting 128

These two extracts above demonstrate the difference in behaviour.

2. Use of “script” to get around bug 857158

Unfortunately the use of “script” in its current form does not work on darwin. The fifth answer on this page suggests an alternative form for the script command to use under darwin - I will try it.