TREC 2021 Fair Ranking Track

Click here to switch to the current Fair Ranking Track webpage.

The TREC Fair Ranking track evaluates systems according to how well they fairly rank documents. The 2021 track focuses on fairly prioritising Wikimedia articles for editing to provide a fair exposure to articles from different groups.

Timeline

Downloads

The TREC 2021 Fair Ranking Track participation guidelines, experimentation protocol, data and evaluation scripts will be available here.

The files are also available from a Globus repository. Most US research universities, and many other institutions, support Globus; you can also download the files with Globus Connect Personal. You can also access the file repository by HTTP through the Boise State data repository.

Resources

Citation

References to the TREC 2021 Fair Ranking Track should use the following citation:

@inproceedings{trec-fair-ranking-2021,
  Author = {Michael D. Ekstrand and Graham McDonald and Amifa Raj and Isaac Johnson},
  Booktitle = {The Thirtieth Text REtrieval Conference (TREC 2021) Proceedings},
  Title = {Overview of the TREC 2021 Fair Ranking Track},
  Year = {2022}
}

If you use a specific dataset, please cite the notebook paper associated with the data.

Organizers

Headshot of Michael Ekstrand
Michael Ekstrand
People and Information Research Team (PIReT)
Boise State University
Headshot of Graham McDonald
Graham McDonald
Information Retrieval Group
University of Glasgow
Headshot of Amifa Raj
Amifa Raj
People and Information Research Team (PIReT)
Boise State University
Headshot of Isaac Johnson
Isaac Johnson
Wikimedia Foundation
Headshot of Morten Warncke-Wang
Morten Warncke-Wang
Wikimedia Foundation