Getting those pesky python modules to work

The python language is very cool because it is easy to use and versatile. Unfortunately, like most languages, it has its quirks. One of the very first problems a newbie will run into importing local python functions and classes. This short guide will help get your project setup correctly for using python modules.

Installing Python

Go to the python website and download the correct python package for your operating system

When installing, make sure to select the ‘Add Python to Path’ option! Otherwise you will not be able to invoke ‘python’ from the command line.

Confirm installation worked by going to the command line and run the following command (case sensitive) which should return the version of python you installed:

$ python -V

Getting Started

Create the following folder structure and files:

Add. /root/source/python/utilities/util.py

Add. /root/entry.py

This is a simple python app that will print out ‘Hello World’ when called. Test it out by running this following from the command line:

$ python ./entry.py

Nerd bit.

After running the python app, a folder called ‘__pycache__’ will be generated under each folder that referenced for import in the python application. This get’s generated to help get the python program start faster. It can be largely ignored and does not need to be committed into source code. In order to suppress the generation of pycache, add the ‘-B’ flag when executing a .py script, for instance:

$ python -B ./entry.py

For more information on pycache, check out this quora answer and stackoverflow answer.

So what’s the problem?

Sometimes we may want to move the entry.py into a different folder (sometimes called ‘dist’ or ‘bin’) to achieve better separation of concerns in our folder structure. Let’s see what happens when we create a folder called ‘dist’ and move entry.py into it.

Move. /root/entry.py to /root/dist/

Now rerun the following command to see if you get the same result.

$ python ./dist/entry.py

Here is what the terminal spat out at me:

The solution is tell python about the full context of your app by letting it know about the root level directory which contains the folder called ‘source’. This can be achieved by adding the following code to the beginning of your entry python file:

import os
import sys

def addDirectoryToPaths(pathToDir, verbose = False):
    #get the immediate parent directory name of this file
    Directory = os.path.dirname(__file__)
    #join the parent directory to the given relative path
    Join = os.path.join(Directory, pathToDir)
    #convert to an absolute path
    Abs = os.path.abspath(Join)
    #add it to the system paths
    sys.path.insert(0, Abs)
    if(verbose):
        print(f"\n\nAdding '{Abs}' to system paths")
        print("\nSystem Paths:")
        for path in sys.path:
            print(path)

addDirectoryToPaths('..')

Edit. /root/dist/entry.py

Try rerunning the following command and watch magic happen!

$ python ./dist/entry.py

Nerd bit.

This is a common problem that people can run into when the entry point for their python app is not at the root project level. This will happen if we moved entry.app into any folder other than the root folder. Python is looking for the ‘source’ folder because we referenced it in our import statement. However, it doesn’t know about it because that folder lives one level above where the entry.py file resides.

The goal is to tell the python system paths about the root project folder where the folder ‘source’ lives. Since the entry.py lives in the bin folder, which is one level away from the root, we pass ‘..’ to the addDirectoryToPaths which tells the function to go up one folder from where we are now.

More Examples

Once you insert the root project level into the system paths, you can import any python script starting from the source file and going all the way down to the script. For instance, add the following folders and files to your project:

Add. /root/source/python/test/otherfolder/myscript.py

Add. /root/source/app.py

Edit. /root/dist/entry.py

Running the application from entry.py will yield:

app running...
myscript.py is running!

What if I want entry.app somewhere else?

The entry python file can be wherever you want in the project, as long as it you change the input to addDirectoryToPaths so that it reaches the root level.

Move & Edit. /root/dist/entry.py to /root/source/python/test

Now that entry.py is three levels deep, we need to change the input to addDirectoryToPaths from ‘..’ to ‘../../..’ telling the function to travel three levels up to the root directory. After that is done, you can get the python app to run by using the same command as before but with a slight change in the path and you will get the same result as before!

$ python ./source/python/test/entry.py