To delete an alert, select it from the list on the left rail, and then hit the red 'delete' button near the top right corner.