The self-hosted Swift Package Manager

4 minute read

Swift Package Manager is itself a swift package containing 6 targets :

libc : Generates a module called libc which is Glibc on linux and Darwin.C on OSX. POSIX : This module Swiftifies the POSIX APIs so that they can be used without all the C like code inside swift. sys : Contains APIs to interact with file system (in swift-like way). PackageDescription : The module we import in our swift packages containing package description, target, dependencies, versions. dep : This is the main module which clones git repos and invokes llbuild process to build the packages. swift-build : And finally, the executable module which parses the input and drives the build process.

llbuild is “A low-level build system, used by the Swift Package Manager”. The swift-build-tool product of llbuild is used by swift package manager to compile the swift packages.

Swiftpm has a python bootstrap script which initially compiles all of swiftpm targets and generates the swiftpm executable. The bootstrapped exectuable is then used to compile the swiftpm source and generate the final swift-build product.

Lets explore a bit of bootstrap script :

Target : A python class which contains name, dependencies, flags etc for a target. As soon as it is initialized, it figures out if a target is a library or an executable by checking for main.swift

class Target ( object ): ... def __init__ ( self , name , dependencies = [], swiftflags = [], extra_libs = [], subpath = None , is_test = False ): self . name = name self . dependencies = list ( dependencies ) self . swiftflags = list ( swiftflags ) self . extra_libs = list ( extra_libs ) self . is_test = is_test # Discover the source files, and whether or not this is a library. self . is_library = True self . swift_sources = [] for ( dirpath , dirnames , filenames ) in os . walk ( os . path . join ( g_source_root , subpath or self . name )): for name in filenames : path = os . path . join ( dirpath , name ) _ , ext = os . path . splitext ( name ) if ext == '.swift' : if name == 'main.swift' : self . is_library = False self . swift_sources . append ( path ) # Exclude tests and fixtures, for now. dirnames [:] = [ name for name in dirnames if name != " " and not name . lower () . startswith ( "fixtures" )] self . swift_sources . sort () ...

It contains a method which writes the .llbuild commands needed for this target to be built to output string buffer.

def write_swift_compile_commands ( self , opts , target_build_dir , module_dir , output , objects , link_input_nodes , predecessor_node ):

The array of Target objects are defined in a targets global variable :

targets = [ Target ( 'PackageDescription' ), Target ( 'libc' ), Target ( 'POSIX' , dependencies = [ "libc" ]), Target ( 'sys' , dependencies = [ "POSIX" , "libc" ]), Target ( 'dep' , dependencies = [ "sys" , "PackageDescription" ]), Target ( 'swift-build' , dependencies = [ "dep" , "sys" , "PackageDescription" , "POSIX" , "libc" ]), ...

When the script is run, the following options are pretty important to it but they’re all filled automatically if toolchain is installed and exported correctly.

swiftc_path : Swift compiler path

sbt_path : Swift built tool path (llbuild)

sysroot : needed on OSX (macosx sdk)

build_path : $pwd/.build dir

parser . add_option ( "" , "--swiftc" , dest = "swiftc_path" , help = "path to the swift compiler [ % default]" , default = os . getenv ( "SWIFT_EXEC" ) or "swiftc" , metavar = "PATH" ) parser . add_option ( "" , "--sbt" , dest = "sbt_path" , help = "path to the 'swift-build-tool' tool [ % default]" , metavar = "PATH" ) parser . add_option ( "" , "--sysroot" , dest = "sysroot" , help = "compiler sysroot to pass to Swift [ % default]" , default = g_default_sysroot , metavar = "PATH" ) parser . add_option ( "" , "--build" , dest = "build_path" , help = "create build products at PATH [ % default]" , default = ".build" , metavar = "PATH" ) parser . add_option ( "" , "--prefix" , dest = "install_prefix" , help = "use PATH as the prefix for installing [ % default]" , default = "/usr/local" , metavar = "PATH" ) parser . add_option ( "-v" , "--verbose" , dest = "verbose" , action = "store_true" , help = "use verbose output" ) parser . add_option ( "--xctest" , dest = "xctest_path" , action = "store" , help = "Path to XCTest build directory" ) parser . add_option ( "" , "--build-tests" , dest = "build_tests" , action = "store_true" , help = "enable building tests" ) opts , args = parser . parse_args ()

the build_path (where swiftpm will be finally built to) is set to .build dir and sandbox_path (where bootstrap binary will be built) is set to .build/.bootstrap

# Compute the build paths. build_path = os . path . join ( g_project_root , opts . build_path ) sandbox_path = os . path . join ( build_path , ".bootstrap" )

The get_swift_build_tool_path() method is used to look for the swift-build-tool (llbuild) if it is not supplied at input

# Determine the swift-build-tool to use. opts . sbt_path = os . path . abspath ( opts . sbt_path or get_swift_build_tool_path ())

create_bootstrap_files() method creates the .llbuild file which will build all targets and the executable.

It uses the write_swift_compile_commands method on target objects.

The file is created at .build/.bootstrap/build.swift-build .

# Create or update the bootstrap files. create_bootstrap_files ( sandbox_path , opts )

Now the bootstrap script is ready to spawn swift-build-tool to compile swiftpm

# Run the stage1 build. cmd = [ opts . sbt_path , "-f" , os . path . join ( sandbox_path , "build.swift-build" )] if opts . verbose : cmd . append ( "-v" ) note ( "building stage1: % s" % ' ' . join ( cmd )) result = subprocess . call ( cmd ) if result != 0 : error ( "build failed with exit status % d" % ( result ,))

process_runtime_libraries method is called on the generated PackageDescription module to check if they’re being loaded correctly or not.

runtime_module_path , runtime_lib_path = process_runtime_libraries ( sandbox_path , opts , bootstrap = True ) def process_runtime_libraries ( build_path , opts , bootstrap = False ):

And finally, the swiftpm builds itself :