(My Notes) How Building a pypi package

Photo by Jiawei Zhao on Unsplash

❓The Problem:

I create a wrapper for the Coin Market Cap API, and I want to use it in another project, but I don’t want to duplicate the files for the new project.

💡The solution:

Create my package, and later use this package in the new project, similar to everyone using the requests package to create HTTP requests in python projects.

🔧The Implementation:

Use `pip` the package manager for Python for I need to create a project and build it. If everything works, I should upload it to pypi.org so I can use `pip install` to install this package and use it in any other project.

💡 This article will use the wrapper I made for the Coin Market Cap API. I’m still working on this wrapper. So this code might change with the time, but the steps to create the package won’t change.

The structure of the project is more or less like this:

coinmarketcap/
├─ src/
│  ├─ cmc_api/
│  │  ├─ __init__
│  │  ├─ cmc.py
│  │  ├─ cmc_datahandler.py
│  │  ├─ cmc_helper.py
│  │  ├─ cmc_utils.py
│  ├─ test/
│  │  ├─ test_files.py
│  ├─ __init__.py
│  ├─ config.ini
├─ .gitignore
├─ README.md

From the tree above:

1. src/ folder contains the main part of the API wrapper.
2. cmc_api/ the folder holds the code for the wrapper.
3. test/ contains the unit test.
4. config.ini is used to store some information like API credits.
5. README.md provides a description of the project. NOTE: this file will provide a long description of the pypi.org package page.

✋The __init__.py files tell python which folders are modules. In most cases, this file will be empty.

Photo by volant on Unsplash

The Steps

1. What files do I need to build the package?
2. What information is in each package?
3. What additional package do I need to build and upload the package?
4. Building, deploying on test.pypi.org.
5. Deploy on pypi.org.
6. Some errors I run into when doing the process.

1. What files do I need to build the package?

I already have a project with a specific structure. I have a README.md file, and the project code is within a folder called /src.

This information will be relevant for the creation of the setup.cfg files.

The files I need for the project will be:

1. README.md
2. setup.cfg
3. pyproject.toml
4. LICENSE
5. (optional) MANIFEST.in.

Now the project will look like this:

coinmarketcap/
├─ src/
│  ├─ cmc_api/
│  │  ├─ __init__
│  │  ├─ cmc.py
│  │  ├─ cmc_datahandler.py
│  │  ├─ cmc_helper.py
│  │  ├─ cmc_utils.py
│  ├─ test/
│  │  ├─ test_files.py
│  ├─ __init__.py
│  ├─ config.ini
├─ .gitignore
├─ LICENSE.json
├─ README.md
├─ pyproject.toml
├─ setup.cfg

I will describe what to add to each file in the next section. For now, I will provide a simple description of what those files do.

README.md

This file is a markdown file, commonly used to provide some information on the GitHub repository page. pypi.org will use the content of this file as a source for the long description of the setup.cfg is configured to do so.

setup.cfg

🔥 There are discussions and resources explaining the usage of setup.py , which is a way to create this setup.cfg file programmatically. I prefer to build the setup.cfg by myself.

This file provides all the details and information of the project to pypi.org.

This information includes:

• Name of the package, what it is used with pip install name-of-the-package.
• Author(s).
• Description: Here is where I will tell Pypi.org the long descriptions in the README.md File.
• Additional package required to use this project (example: request the package needs it in this project since it is the module used to make the HTTP request).
• etc.

Pypi project page

pyproject.toml

It tells build tools (like pip and build) what is required to build your project.

LICENSE

This file tells users who install your package the terms under which they can use the package.

2. What information is in each package?

README.md

This file will be the project’s presentation on GitHub and pypi.org, so it is a good idea to provide information about the usage and design of the project. Below is an example of the main sections for a README.md.

Example of sections in a README.md file

Here is a good template:

GitHub - dbader/readme-template: README.md template for your open-source project

Here is part of what my project README.md look like on Github:

my README.md on Github

setup.cfg

☝🏾 setup.cfg is the configuration file for setuptools. It tells setuptools about your package (such as the name and version) as well as which code files to include.

This file is a configparser format so I should not place quotes around the values, it will have a section defined with [] like [metadata], inside this section, there will be a key-value pair that will provide the information.

setup.cfg file

The example above is the minimal configuration for a project, notice the following:

• [metadata] has a parameter long_description which is using [README.md](<http://README.md>) file, more details about file: later.
• [options] has a parameter package_dir that provides direction to the folder containing the code for the project. Ir is necessary to add where in the section [options.packages.find].

The setup.cfg has a variety of parameters:

[metadata]

• name is the distribution name of your package.
• version is the package version.
• author and author_email are used to identify the author of the package.
• description is a short, one-sentence summary of the package.
• long_description is a detailed description of the package In this case, the long description is loaded from README.md (which is a common pattern) using the file: directive.
• long_description_content_type tells the index what type of markup is used for the long description. In this case, it’s Markdown.
• url is the URL for the homepage of the project. Either a website or a link to GitHub, GitLab, or Bitbucket.
• project_urls lets you list any number of extra links. Generally, this could be to documentation, issue trackers, etc.
• classifiers give the index and pip some additional metadata about your package. see https://pypi.org/classifiers/.

[options]

• package_dir The directory in the project contains all Python source files for the package. An empty package name represents the “root package”. in the example, src directory is designated the root package.
• packages is a list of all Python import packages that should be included in the distribution package. we can use the find: directive to automatically discover all packages and subpackages and options.packages.find to specify the package_dir to use.
• python_requires gives the versions of Python supported by your project.

☝🏾 The directives allowed are file:, attr:, find: and find_namespace: these can be used to instruct setuptools to find some information, for example, the long description or the package will be in a file called README.md therefore long_description = find: README.md

Complex values can be added with comma-separated values or placed one per line.

Complex values

All the parameters available for the [metadata] are:

[metadata] keys and types

And for the [options]

[options] key and types

Here is how my setup.cfg file looks like:

coinmarketcapAPI/setup.cfg at main · CubeVic/coinmarketcapAPI

More details:

https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/

pyproject.toml

The most basic configuration will be:

Where:

• build-system.requires gives a list of packages that are needed to build your package. package here is only used and available during the building process not afterward.
• build-system.build-backend If you were to use a different build system, such as flit or poetry, those would go here, in this case, I am building with the setuptools.

This is my pyproject.toml

coinmarketcapAPI/pyproject.toml at main · CubeVic/coinmarketcapAPI

LICENSE

The best way to generate the license is through the website https://choosealicense.com/

Choose your license

Here how the license looks like:

coinmarketcapAPI/LICENSE at main · CubeVic/coinmarketcapAPI

3. What additional package do I need to build and upload the package?

Build

Once I Create all the files I need the tools for; Building the package and uploading the package.

First, To build the package I can use build

build

pip install build

Twine

Twine is a utility for publishing Python packages on PyPI.

twine

pip install twine

4. Building, deploying on test.pypi.org.

Build

After installation of the package required (build and twine), I can start building the package.

I can run the following command:

python -m build

Building the package

As a result:

1. A folder called dist is created
2. Inside the folder, the builder will save two files, one .whl and the other tar.gz these are the files I need to upload to test.pypi.org and later to pypi.org.

test.pypi.org

This is the test side for the packages, it works similarly to pypi.org however I need a different account to sign up, once the account is activated I can upload the package to this side and proceed to test.

test.pypi.org

Twine

If Twine is already installed I just need the following command to start uploading the files to test.pypi.org.

Using twine to upload the package

python -m twine upload --repository testpypi dist/*

Add the credentials (test.pypi.org and pypi.org provide other means of authentication, for example via access tokens )

Using twine to upload files

The package on test.pypi.org

if I want, I can install the test package and test it.

pip install -i <https://test.pypi.org/simple/> cmc-api-wrapper==0.1.19b0

5. Deploy on pypi.org.

If everything is working and no error stops the function of the code. It is a good idea to test the package first in test.pypi.org. I can proceed to upload it to pypi.org.

I don’t need to build the package again if everything is working, so the only change is where are the packages going to be uploaded, so I need to do some changes to the twine command.

Using twine to upload the package

python -m twine upload --repository pypi dist/*

The package on pypi.org

6. Some errors I run into when doing the process.

Error due to quotes on setup.cfg

During the building of the package I got

Error due to quotes on setup.cfg

I checked the setup.cfg

setup.cfg with quotes in version parameter

The version was version="0.1.10" , with quotation marks, when I remove the " everything work.

Error when assuming build package was already installed

Trying to use python -m build I run into

no package build installed

solve it by installing it with pip install

pip install build

Error requests not part of the build

I had an issue, the request module was not installed or wasn't part of the build, so when I tried to the package, I will get an error that the request package was needed it. I solve it by defining requests as a module on the setup.cfg file.

Adding the request package to setup.sfg

Final Thoughts

1. The upload process can be easier if the access token is generated.
2. It had an additional section on the file pyproject.toml, this is related to the versions, I use bumpver to increment the major, minor, or patch in the version. I don't know how to automate this increment in the versions, therefore, the settings were not included in this article.

Victor Fernandez

• Medium
• My LinkedIn