The Image Embedding Association Test (iEAT) is a statistical test for bias in unsupervised image representations.

Image Embedding Association Test (iEAT)

Ryan Steed, Aylin Caliskan

Read the paper here. View code and data here.

Installation

# gather the dependencies for running scripts in this repo
conda env update environment.yml
conda activate ieat
# install the weat package locally
pip install -e weat
# install the ieat package locally
pip install -e .

Usage

The ieat package does not have a CLI. Use it programmatically by accessing the API module (ieat.api).

To run a basic test on a set of images, use the test function in ieat.api. SimCLR is downloaded automatically - but you must download a pre-trained version of iGPT yourself.

For an example of how to use the API programmatically, see the documentation and tutorials.

Tutorials and Replications

This repo uses Colab scripts in the notebooks/ directory. Check out notebooks/README.md for a full description.

To open a .ipynb file in Colab, navigate to Colab's Github Interface and search for this repo.

Documentation

Documentation for the ieat API is published at rbsteed.com/ieat.

To generate the documentation, use pdoc3:

pdoc3 --html --output-dir docs --force ieat --template-dir docs/templates
git subtree push --prefix docs/ieat origin gh-pages

Contents

• data/ - images and other data used for bias tests in the paper
• embeddings/ - location for caching computed embeddings - includes pre-computed embeddings for convenience; to generate your own, use the from_cache=False option
• ieat/ - software package for generating image embeddings and testing for bias
• notebooks/ - Colab notebooks containing tutorials and data exploration
• output/ - location for storing results tables
• environment.yml - Conda environment file with dependencies for Jupyter, etc.
• docs/ieat - source for documentation

"""
The Image Embedding Association Test (iEAT) is a statistical test for bias in unsupervised image representations.

.. include:: ../README.md
"""

Sub-modules

ieat.api

ieat.models

ieat.utils