NB: Building Your Own Package

Some Examples of Packages

Scikit Learn: Repo | Site (Structure models code)
NLTK Repo | Site (Structure models spaghetti)
SpaCy Repo | Site (API) (Structure kinda models code)
https://github.com/ontoligent/mazo
https://github.com/ontoligent-design/polo2

Why Build a Package?

You package code in order to add it to your python system for general use, and to share it with others.

It’s also good to have well organized code.
This applies to how to organize your programs internally, and externally as files and directories.

It’s easy to do. There are a bunch of nifty tools that help you build, install and distribute packages.

What is a Package?

Again: A collection of modules (python files) along with: * the documentation * the tests * any top-level scripts * any data files required * instructions and scripts to build and install it

What does it mean to build your Package?

To build your own package, you of course need some Python files you want to deploy.

Then you create the following:

Create the basic package structure, such as EXAMPLE 3 below.
Write a setup.py using a package tool (see below).

## EXAMPLE 3

a_package_dir
    __init__.py
    module_a.py
tests
    ...
setup.py # Or pyproject.toml

This will be contained by a project directory.

About `setup.py`

Your setup.py file describes your package, and tells the packaging tool how to package, build, and install it.

It is Python code, so you can add anything custom you need to it.

In the simple case, it is basically a configuration files with keys and values.

What does setup.py contain?

Version & package metadata
List of packages to include
List of other files to include
List of dependencies
List of extensions to be compiled

About pyproject.toml

For a lot reasons that beyond the scope of this document, setup.py
is being superceded by the use of pyproject.toml files to store
setup configuration information.

However, for now we’re going to stick to the old school approach.

Example Setup Files

Example 1

from distutils.core import setup

setup(name='mypkg',
      version='1.0',
      
      # list folders, not files
      packages=['mypkg', 'mypkg.subpkg'], # Include packages in the project
      install_requires=['click'],         # Required libraries
)

Example 2

from setuptools import setup, find_packages

setup(
    name='MyPackageName',
    version='1.0.0',
    url='https://github.com/mypackage.git',
    author='Author Name',
    author_email='author@gmail.com',
    description='Description of my package',
    packages=find_packages(),    
    install_requires=['numpy >= 1.11.1', 'matplotlib >= 1.5.1'],
)

Example 3

from setuptools import setup

setup(
    name = 'PackageName',
    version = '0.1.0',
    author = 'An Awesome Coder',
    author_email = 'aac@example.com',
    packages = ['package_name', 'package_name.test'],
    scripts = ['bin/script1','bin/script2'],
    url = 'http://pypi.python.org/pypi/PackageName/',
    license = 'LICENSE.txt',
    description = 'An awesome package that does something',
    long_description = open('README.txt').read(),
    install_requires = [
        "Django >= 1.1.1",
        "pytest",
    ],
)

A Summary of Keys

As mentioned about, the main content of basic setup files is configuraton information. The keys that you should include in your projects are the following:

name: A string of the package name as title, not a filename.
version: A string of the version number expression, typically using the MAJOR.MINOR.PATCH pattern. See Semantic Versioning for more information.
author: A string with the creator’s name.
author_email: A string with the creator’s email address.
packages: A list of strings of package directories in the project.
url: A string of the URL to the code repo.
license: A string of the license file name.
description: A string with a short blurb of the project.
long_description: A link to a longer description. Can do something like open('README.txt').read().
install_requires: A list of strings of external libraries that the project requires.

Python packaging tools

In writing setup.py, you need to use a packaging tool.

Notice that we’ve imported the setuptools library.

The package tool distutils is included with Python, but it is not recommended.
Instead, use setuptools, a third party tool that extends distutils and is used in most modern Python installations.

Summary of fields

Quick Demo

So, let’s look at a simple package.

Source: Minimal Structure (python-packaging)

Setup file

Here is what out setup.py file has inside:

print(open('demo_package3/setup.py', 'r').read())

from setuptools import setup

setup(name='funniest',
      version='0.1',
      description='The funniest joke in the world',
      url='http://github.com/storborg/funniest',
      author='Flying Circus',
      author_email='flyingcircus@example.com',
      license='MIT',
      packages=['funniest'])

`init.py`

print(open('demo_package3/funniest/__init__.py', 'r').read())

from .funniest import joke

print("Have I got a joke for you!")

`funniest.py`

print(open('demo_package3/funniest/funniest.py', 'r').read())

def joke():
    "This function just tells a joke. Or tries to."
    return (u'Wenn ist das Nunst\u00fcck git und Slotermeyer? Ja! ... '
            u'Beiherhund das Oder die Flipperwaldt gersput.')

Install

!cd demo_package3/; pip install .

Processing /Users/rca2t1/Dropbox/Courses/DS/DS5100/DS5100-2023-07-R/repo/notebooks/M09_PythonModules/demo_package3
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: funniest
  Building wheel for funniest (setup.py) ... done
  Created wheel for funniest: filename=funniest-0.1-py3-none-any.whl size=1568 sha256=aaf3a17b5850b50aa94726bb270982c8c0ace28842225da3d7758da432279f18
  Stored in directory: /private/var/folders/14/rnyfspnx2q131jp_752t9fc80000gn/T/pip-ephem-wheel-cache-c5nojes6/wheels/d2/5e/8e/86778a8bd9f4b020fd7742ecb70af3c25ecae45e348d297469
Successfully built funniest
Installing collected packages: funniest
Successfully installed funniest-0.1

Try it out

from funniest import joke

Have I got a joke for you!

joke()

'Wenn ist das Nunstück git und Slotermeyer? Ja! ... Beiherhund das Oder die Flipperwaldt gersput.'

Many Ways to Install

Running setup.py directly with python

python setup.py sdist   # Builds a source distribution as tar archie
python setup.py build   # Builds from source
python setup.py install # Installs to Python
python setup.py develop # Installs in develop mode (changes are immediately reflected)

Using pip

pip install .    # Installs to Python
pip install -e . # To create symlink, so you can keep working on the code (develop mode)

Testing Code

As you work, you will want to write tests and put them somewhere. You have options:

If your package and test code are small and self contained, put the tests in with the package, e.g. package/test.
If the tests are large or complex, or require reading/writing files, or significant sample data, put your tests outside the package.

See Where to Put Tests?.

A More Complex Package Structure

project_name/
    bin/
    CHANGES.txt
    docs/
    LICENSE.txt
    MANIFEST.in
    README.txt
    setup.py
    test_module_1.py
    test_module2.py      
    package_name/
        __init__.py
        module1.py
        module2.py

CHANGES.txt: log of changes with each release

LICENSE.txt: text of the license you choose (do choose one!)

MANIFEST.in: description of what non-code files to include

README.txt: description of the package should be written in ReST or Markdown (for PyPi):

setup.py: the script for building/installing package.

bin/: This is where you put top-level scripts

( some folks use scripts )

docs/: the documentation

package_name/: The main package this is where the code goes.

test/: your unit tests. Options here:

More Info

https://packaging.python.org/en/latest/overview/