TREC 2021 Fair Ranking Track

The TREC Fair Ranking track evaluates systems according to how well they fairly rank documents. The 2021 track focuses on fairly prioritising Wikimedia articles for editing to provide a fair exposure to articles from different groups.

Tentative Timeline

Downloads

The TREC 2021 Fair Ranking Track participation guidelines, experimentation protocol, data and evaluation scripts will be available here.

The files are also available from a Globus repository. Most US research universities, and many other institutions, support Globus; you can also download the files with Globus Connect Personal. You can also access the file repository by HTTP through the Boise State data repository.

Resources

Citation

General references to the TREC Fair Ranking track should use the following citation:

@inproceedings{trec-fair-ranking-2019,
  Author = {Asia J. Biega and Fernando Diaz and Michael D. Ekstrand and Sebastian Kohlmeier},
  Booktitle = {The Twenty-Eighth Text REtrieval Conference (TREC 2019) Proceedings},
  Title = {Overview of the TREC 2019 Fair Ranking Track},
  Year = {2019}
}

If you use a specific dataset, please cite the notebook paper associated with the data.

Organizers

Headshot of Michael Ekstrand
Michael Ekstrand
People and Information Research Team (PIReT)
Boise State University
Headshot of Graham McDonald
Graham McDonald
Information Retrieval Group
University of Glasgow
Headshot of Amifa Raj
Amifa Raj
People and Information Research Team (PIReT)
Boise State University
Headshot of Isaac Johnson
Isaac Johnson
Wikimedia Foundation
Headshot of Morten Warncke-Wang
Morten Warncke-Wang
Wikimedia Foundation