Creating a Python CLI with Go(lang)-comparable startup times
Hi Folks.

I recently put some effort into creating a command line interface (CLI) made with Python.

Background: I started a new project called Gefyra, a tool for local application development directly with Kubernetes. Check it out the website [https://gefyra.dev](https://gefyra.dev) or have a glance at the code [https://github.com/gefyrahq/gefyra/tree/main/client](https://github.com/gefyrahq/gefyra/tree/main/client)

I'd like to have an executable with (almost) the startup performance of `kubectl` (the executable to control a Kubernetes cluster). That means, I need fast startup times (which is crucial for a CLI) and ideally just one file (which is statically-linked) for easy distribution. In addition, I’d like to provide executables for Windows, MacOS and Linux. For those requirements people would usually go for Go (needless to say it's awesome), however I started out with a prototype written in Python and it evolved over time. So I tried to find a way to make this work with Python.

I went the following way:

1. PyInstaller: [https://pyinstaller.readthedocs.io/en/stable/](https://pyinstaller.readthedocs.io/en/stable/)
2. Nuitka: [https://nuitka.net/](https://nuitka.net/)
3. PyOxidizer: [https://pyoxidizer.readthedocs.io/en/stable/](https://pyoxidizer.readthedocs.io/en/stable/)

**PyInstaller**

PyInstaller was quite easy to set up. However, the resulting executable was complained about by Virustotal (see: [https://www.virustotal.com/gui/home/upload](https://www.virustotal.com/gui/home/upload)) because of PyInstaller's bootloader. Somehow the code signature was also found in viruses (lol). To workaround this I compiled a bootloader myself which at least removed the virus issues. 

On MacOS I faced startup times of more than 10 s with internet connection and about 3 s without internet connection. Interestingly, the former docker-compose command was also created from PyInstaller and Mac users complained about the startup performance, too: [https://github.com/docker/compose/issues/6956](https://github.com/docker/compose/issues/6956) :)

I didn’t find much to improve. The concept of PyInstaller will potentially always be a problem for fast startup times (which IMHO makes it unsuitable for CLI applications).

**Nuitka**

With Nuitka, I generated very large binaries of about 150 Mb. The startup performance was already much better than PyInstaller for Mac and Linux. However, I was not completely satisfied and very long compile times bothered me a little bit (about 10 min).

**PyOxidizer**

I ended up using PyOxidizer. This well-crafted toolkit compiles Python to Rust code and also includes all dependencies into one handy binary executable. With no special optimizations I saw startup times of about 700 ms. That is almost acceptable, though I wanted to go a little further.

I started to examine the output of `python -X importtime -m gefyra 2&gt; import.log` just to check the imports. There is an awesome tool to analyze the Python imports: tuna (see: [https://github.com/nschloe/tuna](https://github.com/nschloe/tuna)). `tuna` allows analyzing the import times from the log. Run it like so `tuna import.log`. It opens a browser window and visualizes the import times. With that I was able to manually move all imports to the functions in which they are needed (and bring in some other optimizations). This greatly violates PEP 8 ([https://peps.python.org/pep-0008/#imports](https://peps.python.org/pep-0008/#imports)) but leads to very fast startup times.

These are the startup values I finally reached with `gefyra` under average modern Ubuntu:

    &gt; python -m timeit "__import__('os').system(gefyra)"  
    10 loops, best of 5: 33.5 msec per loop  

Pretty neat, isn’t it?   
In comparison the `kubectl` executable:

    &gt; python -m timeit "__import__('os').system('kubectl')"  
    10 loops, best of 5: 24.9 msec per loop  

In addition, I created GitHub actions to run the PyOxidizer builds once a new version is released (see: [https://github.com/gefyrahq/gefyra/blob/main/.github/workflows/dist-build-linux.yaml](https://github.com/gefyrahq/gefyra/blob/main/.github/workflows/dist-build-linux.yaml)). Only Windows is missing at the moment.

Although, PyInstaller and Nuitka did not deliver the best startup times, I would not say it's bad software. They probably shine at other aspects.

I hope these insights can be useful for someone else, too.