Submissions

Please prepare your results in a CSV file, to be emailed to roman_data_challenge_submissions@stonybrook.edu. In the email subject, please include which rung the submission corresponds to. In the email body, please include the names, emails, and affiliations of all team members associated with the submission. The submitted CSV file should be named {team name}_rung{rung number}_submission{submission number}.csv. For example, if team "bob" wishes to re-submit for rung 1 due to a bug that appeared in the previous submission, their csv file should be named bob_rung1_submission2.csv.

For rungs involving classification, the models' accuracy, precision, true positive rate (recall), and false positive rate will be evaluated. For rungs involving regression, the grading scheme is identical to that of Ding et al. 2021, section 2.7. For transparency, the notebooks used to grade the submissions are shown here.

Rung 0

In this rung, you will train a regression model to determine the Einstein radius of lenses.

The submitted CSV file should contain the parameters ID, theta_E, and theta_E_sigma in the header row, with each row afterwards corresponding to the results for each of the lenses.

Rung 1

The goal of this rung is to distinguish between mass distributions with Cold Dark Matter subhalos and no subhalos. In this rung, you will train a binary classifier to determine whether subhalos are present.

The submitted CSV file should contain the parameters "ID", "has_substructure" in the header row, with each row afterwards corresponding to the results for each of the lenses.

Rung 2

TBA

Rung 3

TBA