Reviewer-article assignment and article scoring based on Bayesian correction

Reviewer-article assignment and article scoring based on Bayesian correction


website

In our most recent research, we have developed new methods to completely automate the reviewer-article assignment by using topic modeling. The system only needs the article’s abstracts and samples of abstracts written by the reviewers. This removes the biases introduced by either authors suggesting reviewers or reviewers bidding for their favorite articles to review.

Also, we have develop a new method for estimating the underlying score of an article based on judgments of a set of reviewers. Our method goes beyond a simple average of the reviewers’ scores, and it controls for the variances and biases of the reviewers. Sometimes reviewers are too harsh or too nice compared to other reviewers and the system regresses their scores to the mean accordingly. Also, some reviewers are very variables in their reviews or others are very consistent with others, and our system weighs their scores accordingly.

The web implementation of this research has been possible with the help of a new graduate student in the lab, Titipat Achakulvisut, and the web developer Tulakan, and of course, Konrad.

Categories

  • Titipat Achakulwisut

    Wow! This website looks great