Setting up a development environment

Required dependencies

Support matrix 1

Python Version Python 3.9 Python 3.10 Python 3.11
Operating System
Linux 2

macOS (x86_64)

macOS (aarch64)

Windows

  1. Install Miniconda

  2. Install gh

    conda install -c conda-forge gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Miniconda environment using environment.yml

    cd ibis
conda env create --file conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  1. Activate the environment

    conda activate ibis-dev

  1. Install your local copy of ibis into the Conda environment

    uv pip install -e .

  1. Install Mamba

  2. Install gh

    mamba install -c conda-forge gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Mamba environment using environment.yml

    cd ibis
mamba env create --file conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

  1. Activate the environment

    mamba activate ibis-dev

  1. Install your local copy of ibis into the Mamba environment

    uv pip install -e .

  1. Install Pixi

  2. Install gh

    pixi global install gh

  3. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  4. Create a Pixi environment using environment.yml

    cd ibis
pixi init --import conda/environment.yml

    If you’re using arm64 architecture (Mac M1/M2), use conda/environment-arm64.yml for setting up a dev environment for all the backends that are possible to install excluding Flink; use conda/environment-arm64-flink.yml for setting up a dev environment for all the backends that are possible to install including Flink. The reason to have two separate environments is because apache-flink forces pyarrow to downgrade to 11.0, which causes conflicts in other backends.

Note

This command will initialize a pixi.toml file and also modify the .gitignore to prevent Pixi configuration from being added to git.

  1. Activate the environment

    pixi shell

  1. Install your local copy of ibis into the Pixi environment

    uv pip install -e .

Support matrix

Python Version Python 3.9 Python 3.10 Python 3.11
Operating System
Linux 3

macOS (x86_64)

macOS (arm64/M1/M2) 4 5 6
Windows 7 8 9

  1. Install nix

  2. Configure nix

    Edit/create your nix.conf file ($XDG_CONFIG_HOME/nix/nix.conf) and add the line

    experimental-features = nix-command flakes

  3. Install gh:

    nix-shell -p gh
    nix-env -iA gh

  4. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  5. Set up the public ibis Cachix cache to pull pre-built dependencies:

    nix-shell -p cachix --run 'cachix use ibis'

  6. Run nix develop in the checkout directory:

    cd ibis
nix develop

    This will launch a bash shell with all of the required dependencies installed. This may take a while due to artifact download from the cache.

pip will not handle installation of system dependencies

pip will not install system dependencies needed for some packages such as psycopg2 and kerberos.

For a better development experience see the conda/mamba or nix setup instructions.

  1. Install gh

  2. Fork and clone the ibis repository:

    gh repo fork --clone --remote ibis-project/ibis

  3. Change directory into ibis:

    cd ibis

  4. Install development dependencies

    This will also install Ibis in development mode as ibis-framework.

    pip install 'uv>=0.4.29'
pip install -r requirements-dev.txt

Code style and formatting

Ibis uses several code linters and has a style guide for documentation, please checkout the style and formatting guide for instructions on how to set things up.

Building the docs

Install just (if you installed via conda/mamba you are covered) and run

just docs-preview

to build and serve the documentation.

Back to top

Footnotes

  1. Some optional dependencies for Windows and Mac OS are not available through conda/mamba↩︎

  2. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  3. Tested in CI. If this doesn’t work for you, please file an issue.↩︎

  4. Not working as of March 2024. See https://github.com/ibis-project/ibis/issues/8696↩︎

  5. Not working as of March 2024. See https://github.com/ibis-project/ibis/issues/8696↩︎

  6. Not working as of March 2024. See https://github.com/ibis-project/ibis/issues/8696↩︎

  7. Unlikely to ever be supported or no upstream support.↩︎

  8. Unlikely to ever be supported or no upstream support.↩︎

  9. Unlikely to ever be supported or no upstream support.↩︎