@Manizuca,

Manizuca https://forum.f-droid.org/u/manizuca

December 1 nalexander: Can you explain what this means? Do binaries mean things like proprietary

codec blobs? I didn’t see anything like this when I looked in the F-Droid

build scripts at https://gitlab.com/fdroid/fdroiddata/blob/master/

metadata/org.mozilla.fennec_fdroid.txt#L3959-3989. Yeah, the build script calls a script

https://gitlab.com/Manizuca/fenneclocales/raw/master/prebuild.sh within

the same repo that holds the locales. These files are removed because the

fdroid-server checks for the presence of any executable/binary data.

Oh my. In general, this approach is always going to be broken, and Mozilla

would much prefer to hear about and potentially accommodate your use cases

than have you require such blunt instruments. But this approach does have

the advantage of not requiring upstream changes, which I understand!

nalexander:

More details would be helpful here. I would expect that --disable-tests

in your mozconfig wouldn’t fail the builds, but I don’t know what’s at

play. If i remove just the files, but keep the .ini s, i get something like 0:25.26 ==============================

0:25.26 ERROR PROCESSING MOZBUILD FILE

0:25.26 ==============================

0:25.26 The error occurred while processing the following file or one of

the files it includes:

0:25.26 /Sources/F-droid/build/org.fdroid.fennec/modules/libjar/

zipwriter/moz.build

0:25.26

0:25.26 The error occurred when validating the result of the execution.

The reported error is:

0:25.26 Error processing test manifest file /Sources/F-droid/build/org.

fdroid.fennec/modules/libjar/zipwriter/test/unit/xpcshell.ini: Traceback

(most recent call last):

0:25.26

0:25.26 File “/Sources/F-droid/build/org.fdroid.fennec/python/mozbuild/mozbuild/frontend/emitter.py”,

line 1294, in _process_test_manifest

0:25.26 path, ', '.join(missing)), context)

0:25.26

0:25.26 SandboxValidationError:

0:25.26 ==============================

0:25.26 ERROR PROCESSING MOZBUILD FILE

0:25.26 ==============================

0:25.26 The error occurred while processing the following file or one of

the files it includes:

0:25.26 /Sources/F-droid/build/org.fdroid.fennec/modules/libjar/

zipwriter/moz.build

0:25.26

0:25.26 The error occurred when validating the result of the execution.

The reported error is:

0:25.26 Test manifest (/Sources/F-droid/build/org.

fdroid.fennec/modules/libjar/zipwriter/test/unit/xpcshell.ini) lists test

that does not exist: test_asyncadd.js, test_asyncbadadd.js,

test_asyncbadremove.js, test_asyncremove.js, test_bug399727.js,

test_bug419769_1.js, test_bug419769_2.js, test_bug425768.js,

test_bug433248.js, test_bug446708.js, test_bug467740.js,

test_createempty.js, test_deflatedata.js, test_directory.js,

test_editexisting.js, test_storedata.js, test_sync.js, test_undochange.js,

test_zipcomment.js, test_zippermissions.js, test_bug717061.js,

test_alignment.js And if i remove all the directory with the conflicting data including the

.ini s, the error is: 0:23.25 ==============================

0:23.25 ERROR PROCESSING MOZBUILD FILE

0:23.25 ==============================

0:23.25 The error occurred while processing the following file:

0:23.25 /Sources/F-droid/build/org.fdroid.fennec/modules/libjar/

zipwriter/moz.build

0:23.25

0:23.25 The error was triggered on line 7 of this file:

0:23.25 XPCSHELL_TESTS_MANIFESTS += [‘test/unit/xpcshell.ini’]

0:23.25

0:23.25 An error was encountered as part of executing the file itself. The

error appears to be the fault of the script.

0:23.25

0:23.25 The error as reported by Python is:

0:23.25 [‘IOError: Missing files: /Sources/F-droid/build/org.

fdroid.fennec/modules/libjar/zipwriter/test/unit/xpcshell.ini

’] In this example happens with a xpcshell.ini file, but the same happens

with something like MOCHITEST_MANIFESTS += [‘test/mochitest.ini’]. I

opted to remove all the test directories with conflicting files, and then

grep the moz.build to remove the .ini s

Why is it that you need to do anything with these test files? They don’t

feature in the packaged artifacts at all, and if you use --disable-tests

in your mozconfig you should not really see any interaction with them.

nalexander: These should all be supported as mozconfig options. If they’re not doing

the right things for you, I consider that a bug, and would appreciate being

CCed on a ticket in the Core :: Build Config component at

https://bugzilla.mozilla.org. In particular, Telemetry is growing in

scope and intrusiveness, so I wouldn’t be surprised if we need a little

focused attention to ensure we’re not accidentally pushing bits to Mozilla

that F-Droid users don’t want exposed. There was some problems with telemetry code being included even when

disabling it with .mozconfig. I see that many of them were fixed in the

last versions. nalexander: Firefox for Android has been pretty focused on using newer technologies to

A/B test and really understand our user base, so we’ve used Adjust (a

German tracker with a good privacy policy), Switchboard (an A/B module that

we run an in-house backend for), and Leanplum (a marketing application

about which I know little). It’s my belief that we’ll be able to make

building with Google Play Services imply no Adjust and Leanplum. I don’t

know what the right outcome for Switchboard is for F-Droid – you don’t want

Mozilla’s instance, but you probably don’t want to run your own (which has

a non-trivial cost). I’m pretty sure that everything will work if you “opt

out” of Switchboard, but I don’t know if that’s easy to arrange at

build-time. We can try to make it so. Definitely i have to test the last versions, but with a packet analyzer we

discovered some time ago that some information was being sent even when

opting out from the user preferences. As i said, the technology used by

firefox for android has changed a lot since then.

I would appreciate you revisiting, if and when you can, but I don’t think

this should be high priority for you.

nalexander: Surely the majority of F-Droid’s packages are built with Gradle, so this

problem must be addressed in some way. Firefox for Android’s Gradle

configuration is pretty complex, but it should “just work” to run mach

build after the dependencies are in place. I expect that F-Droid could

work through these dependency issues fairly easily. I can provide links to

what Firefox for Android does: the tl;dr is that we collect Gradle

dependencies using a proxy Maven repository running Nexus Sonatype. It’s

not that complicated; start reading at https://searchfox.org/mozilla-

central/source/taskcluster/scripts/misc/android-gradle-dependencies.sh

and the linked scripts. I will try a gradle build with more recent versions (last time i tried was

Fx54), but if you use a custom maven repo it could be problems as there are

just a couple of them whitelisted. @hans

https://forum.f-droid.org/u/hans should know more.

Mozilla does not run its own Maven repository, and is unlikely to depend on

one anytime soon: it’s a non-trivial cost when you run continuous

integration at the scale that we do. The set of Maven dependencies evolves

slowly, so when it changes we capture the entire dependency set, archive

it, and offer that to subsequent builds. Right now, those dependencies are

fetched from jcenter (run by Bintray) and maven.google.com (run by

Google). It might be possible to avoid Google’s repository right now, but

the evolution is definitely towards everything Android related being from

there so it’s probably not possible to avoid for long.

In any case, I don’t think you would have big problems doing what we do;

and, if you so chose, you could also use Mozilla’s dependency archives –

they’re just .tar.xz files of Maven file repositories.

Firefox 58 will probably be the last Firefox for Android that will not

require Gradle to build – watch

https://bugzilla.mozilla.org/show_bug.cgi?id=1414415

to see when the non-Gradle build is removed. In order to have a smooth

transition as Firefox 58 becomes release, F-Droid will need to do some work

to support this.

nalexander: Thanks for your feedback!

Nick Many thanks to you! [image: ]

Is it easy to test changes to F-Droid’s build script? I think we might be

able to do a little focused work and get a much simpler script that

achieves the same outcomes.

Best,

Nick