Welcome to hip-data-tools’s documentation!

© Hipages Group Pty Ltd 2019

PyPI version CircleCI

Common Python tools and utilities for data engineering, ETL, Exploration, etc. The package is uploaded to PyPi for easy drop and use in various environments, such as (but not limited to):

  1. Production workloads
  2. Python notebooks
  3. Local dev and exploratory analysis

Installation

Install from PyPi repo:

pip3 install hip-data-tools

Install from source

pip3 install .

Install using setup tools

python3 setup.py install

Run unit tests using setup tools .. code:: bash

python3 setup.py test

The hip-data-tools package is structured to be used in three different modalities:

  • Use base level Utility classes to interact with services (Modules with the names of their service vendors, like aws, google)
  • Use pre compiled scripts and modules that can be executed from command line (hip_data_tools.etl package)
  • Higher level etl classes, that allow you to build complex data transformations (hip_data_tools.hipages package)

Indices and tables