Import python modules straight from github



In Go we have ability to import modules from github, like:

import "github.com/parnurzeal/gorequest"

It’s a bit controversial feature, but sometimes it’s useful. And I was interested, is it possible to implement something like that in python. TLDR it’s possible with import_from_github_com package:

from github_com.kennethreitz import requests assert requests . get ( 'https://github.com' ). status_code == 200

So, how it works, according to PEP-0302 we have special sys.meta_path with importer objects and every importer should implement finder protocol with find_module(module_name: str, package_path: [str]) -> Loader|None . Now we need to implement finder that handles modules, which path starts with github_com , like:

class GithubComFinder : def find_module ( self , module_name , package_path ): if module_name . startswith ( 'github_com' ): return GithubComLoader () sys . meta_path . append ( GithubComFinder ())

And now we need GithubComLoader that implements loader protocol with load_module(fullname: str) -> None , I’ll skip private methods of the loader here, they’re straightforward and not interesting in context of the article:

class GithubComLoader : def load_module ( self , fullname ): if self . _is_repository_path ( fullname ): self . _install_module ( fullname ) if self . _is_intermediate_path ( fullname ): module = IntermediateModule ( fullname ) else : module = self . _import_module ( fullname ) sys . modules [ fullname ] = module

So what’s IntermediateModule , it’s a dummy module/package for paths like github_com.nvbn , it’s used only in intermediate steps and shouldn’t be used by end user. Installation happens in _install_module method, it just calls pip with git url, like:

import pip pip . main ([ 'install' , 'git+https://github.com/kennethreitz/requests' ])

All looks very simple, let’s try it in action:

>>> from github_com.kennethreitz import requests Collecting git + https : // github . com / kennethreitz / requests Cloning https : // github . com / kennethreitz / requests to / tmp / pip - 8 yyvh7kr - build Installing collected packages : requests Running setup . py install for requests Successfully installed requests - 2.9 . 1 >>> requests . get ( 'https://github.com' ). status_code 200

Sources on github.