If you go into Criteria, the cancel button is disabled.

Once the Cancel button is disabled, you need to fill in the required fields, then hit save, then you can delete the new row.

Note: To avoid the rule running accidentally, before you hit save, UN-select the 'Rule Enabled', that way you won't have anyone run the incorrect/unfinished rule in-between the time you save it and you delete it.