Tools to backup TA partition (drm keys) of Xperia XZ1 Compact



by j4nn

https://j4nn.github.io/

by j4nn

Anybody who is about to unlock your phone, could you please do so with additional test included?

See post#500 and post#502 for more details.

Additional details in post#515, post#516, post#517 and post#527.

Instructions for the test that I kindly ask anybody who is about to unlock to do are described in the post#520 -- tested already.

Thank you.

ABOUT THE TOOLS

renosploit - rename/notify exploit to get kernelspace read/write, uses multiple vulnerabilities to overcome kaslr, pxn and pan mitigations of android oreo

- rename/notify exploit to get kernelspace read/write, uses multiple vulnerabilities to overcome kaslr, pxn and pan mitigations of android oreo renotrap - helper application (rename/notify temp root app)

- helper application (rename/notify temp root app) renoshell - get temp root shell by use of kernel space read/write primitives provided by renosploit (sources available here)

- get temp root shell by use of kernel space read/write primitives provided by renosploit (sources available here) renoroot - a shell script to be started from adb, it starts the above tools to get temp root shell

SUPPORTED TARGETS

Sony Xperia XZ1 Compact (G8441)

47.1.A.2.324_CE1 (initial tested by @tramtrist, this release tested by @tanapoom1234 post#212)

47.1.A.8.49_CE1 (tested by @notaz post#224 and @orsonmmz post#232)



47.1.A.2.324_CE1 (initial tested by @tramtrist, this release tested by @tanapoom1234 post#212) 47.1.A.8.49_CE1 (tested by @notaz post#224 and @orsonmmz post#232) Sony Xperia XZ1 (G8341/G8343)

47.1.A.2.324_CE1 (tested by @HandyMenny post#228)

47.1.A.2.324_CE1 (tested by @HandyMenny post#228) Sony Xperia XZ1 Dual (G8342)

47.1.A.2.281_CE1 (tested by @Vildanoff post#230)

47.1.A.2.281_CE1 (tested by @Vildanoff post#230) Sony Xperia XZ1 (SOV36) /added on 2019-08-22/

this Japan version can be flashed with fw for G8431 making it exploitable as standard XZ1 (the possibility to use G8431 fw is confirmed here and also here)

/this confirms there might be a possibility of TA backup for few yoshino platform phone models that are possible to flash with one of the above firmwares (and boot ok even though designed for other phone variant)/



Sony Xperia XZ Premium (G8141)

47.1.A.3.254_CE1 (tested by @DocLM post#227, by @LinFan post#242 and by @steso90 xzp forum post#45)

47.1.A.3.254_CE1 (tested by @DocLM post#227, by @LinFan post#242 and by @steso90 xzp forum post#45) Sony Xperia XZ Premium Dual (G8142)

47.1.A.3.254_RU (tested by @greatpatel007 xzp forum post#31 and #39)

47.1.A.3.254_RU (tested by @greatpatel007 xzp forum post#31 and #39) Sony Xperia XZ Premium (G8188) /added on 2019-04-24/

this Japan version can be flashed with fw for G8141 making it exploitable as standard XZp (tested by zatsune as documented here)

/this confirms there might be a possibility of TA backup for few yoshino platform phone models that are possible to flash with one of the above firmwares (and boot ok even though designed for other phone variant)/

if flashing a fw for different phone model, skip flashing bootloader

or other kernel builds

USING THE TOOLS

backup everything you need from your phone flash compatible firmware

Before flashing, you may take a screenshot of service menu -> service tests -> security possibly together with current sw version screen for reference and copy them from the phone to your PC.

You can use newflasher tool from @munjeni and use instructions there to flash the firmware.

The tool should skip dangerous .ta files automatically. You may consider removing Just remove the persist_X-FLASH-ALL-42E5.sin file, which is discussed here to avoid flashing it - as tested by @tanapoom1234, not flashing the persist partition allows to keep the Android Attest Key - check his post#212. /Added on 2019-04-06: The key is not part of TA obviously, it is present in the persist partition, so never flash persist even after TA backup./

/Added on 2019-04-09: When flashing a firmware, be sure to flash it's bootloader too (i.e. the whole 'boot' directory needs to be present with all files in it including the .ta there). You might skip appslog, diag, Qnovo and ssd./

In case of downgrade it is needed to flash userdata (and possibly also cache) otherwise you get a boot loop.

Just backup your stuff before downgrade as with downgrade comes a factory reset. In fact I would recommend to do a factory reset just before the downgrade in order to remove the binding to your google account. This way you can avoid going online after the downgrade if used without sim and skipping wifi configuration. prepare your phone

When the phone boots up, try to avoid connecting to internet by selecting only wifi and not configuring any, skipping accounts setup for later.

This may not always be possible - if persist is not flashed, android insists on setup of google account online, also starting downloads for upgrade.

Cancel everything as soon as possible and disable wifi. You may be better not using a data enabled sim card - we try to avoid any updates.

Disable auto updates of both apps and system. Change the theme from animated backgroud to a static one.

Enable developer menu, enable adb and "Stay awake" option. An youtube video showing the initial setup to prepare for renoroot is available here.

Take a screenshot of service menu -> service tests -> security for reference and copy it from the phone.

Again be sure both wifi and mobile data connection are disabled to avoid any background internet access. install the tools

Unzip renoroot.zip (download it bellow). Use following adb commands to get the tools to the phone:

Code: adb push renoroot /data/local/tmp adb push renoshell /data/local/tmp adb push renosploit /data/local/tmp adb install -r renotrap.apk start the tools to get a temp root shell

Use adb shell to get a command line terminal to the phone and use following commands:

Code: cd /data/local/tmp chmod 755 reno* ./renoroot

It may get the phone to reboot in case an overwrite does not hit the wanted shaped heap object.

You may wait few minutes after the phone boots to allow startup processes to settle down in order to avoid timing influence for next trial.

There is a video for example of this step available here. backup your TA partition

When renoroot is successful, you may use following commands in the root shell to backup the trim area partition:

Code: cd /data/local/tmp dd if=/dev/block/bootdevice/by-name/TA of=TA-locked.img chown shell:shell TA-locked.img sync sync

Code: adb pull /data/local/tmp/TA-locked.img unlock phone's bootloader using a code from sony

When you have the TA-locked.img on your PC including screenshots, you may start the official Sony unlock procedure - follow instructions on sony website please.

Added on 2019-04-16: please note, bootloader unlocking is not reversible - it is not possible to re-lock back (restore of TA-locked does not relock the bootloader).

So be prepared to live with the boot up warning screen (can be seen for example in this video).

Again be sure you have the TA-locked.img on your PC before you start unlocking the bootloader - unlock will erase you phone, so it would get lost from /data/local/tmp if not backed up.

In case oem unlocking is grayed out (so you cannot enable it) you need to go online at least once and the option would be accessible then - video here.

After you unlock the bootloader, do not flash anything - just boot the same unmodified fw we used for the temp root. get temp root again to restore TA

Use the same instructions to avoid internet access and updates as described above, configure the few above mentioned options and start renoroot as before.

With the temp root shell, backup the unlocked TA (for future comparisons) and then restore the state from the locked one. You may need to adb push the TA-locked.img back to /data/local/tmp as the unlock erased everything.

Code: cd /data/local/tmp dd if=/dev/block/bootdevice/by-name/TA of=TA-unlocked.img chown shell:shell TA-unlocked.img sync sync

Code: adb pull /data/local/tmp/TA-unlocked.img adb push TA-locked.img /data/local/tmp

Code: cd /data/local/tmp dd if=TA-locked.img of=/dev/block/bootdevice/by-name/TA sync sync boot up the phone with the current fw and see about the camera if it works on not

You may also document the security screen state by taking a screenshot. Do not forget to transfer it from the phone to PC. flash twrp recovery

Updated on 2019-08-08: please see post#1029 for the latest workflow with the kernels hiding bootloader unlock status.

Updated on 2019-02-10:

Instead of flashing twrp, you may just 'fastboot boot' it if you need it.

Instead of the steps 10. to 13., you may use patched and rooted kernel hiding bootloader unlock available in following forum threads in order to be able to even install FOTA system update

[XZ1c] rooted kernel hiding bootloader unlock with working fota

[XZ1] rooted kernel hiding bootloader unlock with working fota

[XZp] rooted kernel hiding bootloader unlock with working fota giving you back sony drm functionality that fw disables when it detects unlocked bootloader status. For more details see also post#645 of this thread. OPTIONAL step (only for XZ1c maybe XZ1)

This step is optional and only lightly tested. The idea is that secd detects unlocked bootloader and switches to limited mode even though drm keys are available. This can be seen in the adb logcat with following message:

Code: E secd : secd_backend_credential_manager.cpp:77 the bootloader is unlocked, use limited functionality secd extension by modpunk - just flash attached secd-ignore-unlock.zip at bottom of this post via twrp recovery (do not flash the 'secd extension by modpunk' which is linked here only for reference).

I've analysed, what changes were done in the secd. Also the lib which fixes the missing device key in TA is not needed from the modpunk's package as we have the real valid key there, so I've removed the lib (and the script which would preload it). Therefore it is just about making secd think that bootloader was not unlocked. Thanks to @modpunk for the patched secd and @russel5 for the flashable zip on which the secd-ignore-unlock.zip is based on.

With this, sony updates may start to arrive.

Please note, this would make sony think the phone runs unmodified and still locked fw. OTA updates may restore original secd or fail altogether (due to modified system/vendor/... partitions).

You may boot the phone to see what happens (OTA updates?) - edit: OTA updates did come, but install to be done on reboot failed - tested by @Unbounded, see post#43 and #44 of the attest key thread please - this may confirm the availability of the SOMC Attest Key which may be the key needed to get sony ota updates (just a guess, not sure what exactly this key is used for) .

Again, this step is optional and very experimental, maybe better not to apply it (camera works without this step on any stock fw without any change /until sony changes that in some update/).

Update: see post#395 for secd_ignore_unlock for XZ1c for pie from @S-trace - thank you. It works with XZ1 too (see post#396). The patch port for XZp pie is here: attest key thread post#67.

In my opinion all these secd patch variants are hiding the unlocked state only partially. There are other components in the fw that ask about the unlock state. A proper solution for this is the unlock hiding patched kernel linked in the step 10. of this howto. flash a recent stock firmware

In case you wanted the patched secd, flash it again over the flashed fw.

Boot the phone, check functionality, take screenshots. install magisk if rooted phone is what you need; -)

Follow instructions of latest magisk, it should work without any special actions.

AUTOMATED FULL BACKUP

and actually seem not to work in some cases getting truncated files that are useless

Code: cd /data/local/tmp ./backup-send.sh

Code: backup-recv.bat bk-unlocked

WHAT WORKS

Quote: Originally Posted by tramtrist Originally Posted by I'd like to report in real quick on what's working.

After following @j4nn very clear instructions and backing up/restoring my TA keys I was left with the NOT PROVISIONED messages he mentioned earlier. However this seems to be no problem as after TA-restore my camera works as it did before. I'm also able to use WIDEVINE sites which require that key as well.

After restoring TA I went ahead and flashed the latest UK customized firmware

I then flashed TWRP latest version 3.2.3

I wanted to have root so I flashed Magisk 1.73 and safety net worked without me having to do anything special.

Google Pay could be set up and seems to be using my credit cards just fine.

I didn't flash any custom kernel as stock is just fine for me.

Adaway is working with root without issue.

All-in-all if you follow @j4nn instructions when he's ready to fully release them to the public then I'd say you will be in good shape.

I'd like to thank @j4nn for giving me the chance to finally contribute something concrete to this community. If you're gonna use this you should drop him some cash.

ACKNOWLEDGEMENTS

@moofesr - for testing initial kernel builds until proper build procedure had been found, special thanks for his patience when all tests resulted with bootloop

@Raz0Rfail and @moofesr - for testing timing of rename/notify vulnerability with patched kernel

@dosomder (aka zxz0O0) - for his iovyroot

@tramtrist - for initial testing of TA backup, unlock and restore, special thanks for exposing to risk of loosing drm if it did not work

@tonsofquestions - for a lot of testing with unlocked-ta-restored phone when I did not have an unlocked phone yet

ThomasKing (not a user on xda) - for his black hat ksma presentation

few other users in this and attest key lost thread here on xda - for some other cve possibilities, ideas and specific tests

DONATIONS

DOWNLOAD THE TOOLS

As everyone knows, bootloader unlock via code from sony removes drm keys. That disables certain functions, the most critical one being the camera (outputting only solid green pictures in case of oreo fw).I've implemented tools that allow to backup the whole TA partition, which contains device master key needed to access sony drm keys and restore the TA after bootloader unlock in order to make the camera (among other things) working again on any sony stock firmware.In order to be able to use the tools, you need to flash one of the supported firmwares (or be lucky to have the phone already running it).In case you need to downgrade, please check this thread first.A preview video of the tools in action can be downloaded here: renoroot-preview.zip or watched online here /added on 2020-02-08/(with downloadable firmware links)/added on 2019-08-27/Please note: the temp root exploit (all renoroot tools) are designed only for the above firmware versions (binary kernels builds in them) - there is no chance it would work on other phones- do not try it, it would not work.Concerning portability to other targets, the exploit itself needs several vulnerabilities not fixed in a kernel, the primary one is CVE-2017-7533 (race between inotify and rename).This was patched by google with 2017-12-05 security patch level . That means unless you can flash a firmware with older security patch level, it would not make sense to try to adapt the exploit for a new target (like it is a case with XZ2 Compact device for example).Please follow the steps bellow for a official and up to date guide. If something was not clear enough, you may also check post#382 from @ munted for a pdf guide with screenshots possibly containing more details and windows specific hints.These are experimental tools () to extract most of the partitions from the phone after getting a temp root. It can be used for comparisons/analysis of what unlock changes (download backup-tools.zip at bottom of this post).You would run backup-setup.bat in windows command prompt first (you may need to adjust the PATH setting to find adb properly) to copy the tools to the phone and setup tcp forwarding for netcat based copying.Then using adb shell you would do:and in windows command prompt you would start:and partitions images would be extracted from the phone (for larger ones sparse android image format is used).Full depth comparison could be achieved by use of these backup tools (obviously needs to be done twice - before and after unlock, changing the target directory name argument of backup-recv.bat).Here is a quote of post#185 from @ tramtrist in this thread describing the results of the initial tests - special thanks to him!Many thanks to following users:Please note: I had to invest enormously lot of time (as you can see throughout this thread and also summarized in progress/change log in post#2 ) to develop these tools, the code is extremely complex (more than 9000 lines of source code) and it was unbelievable hard to debug and get the timing usable.It would be kind of you if you could consider donating here please:I would be happy to accept any donation to me as a form of gratitude in case the software helped you to backup your TA (drm keys) before bootloader unlocking.Thanks.See the attached renoroot.zip at bottom of this post.Please post your experience with using the tools, if it worked and on which phone model (and fw in case of xz1c).You may include info about how long it took to get a root shell, how many reboots, how many events in the last trial which succeeded with how many overwrites (just one with success is the best, more means previous overwrites did not hit wanted object in shaped heap resulting with possibly unstable system). This info is interesting for statistics, so we all know, how fast can we get a temp root on each device/firmware.Thanks.