Computational methods for automated genome annotation are critical to our 
community's ability to make full use of the large volume of genomic sequence 
being generated and released. To explore the accuracy of these automated feature 
prediction tools in the genomes of higher organisms, we evaluated their 
performance on a large, well-characterized sequence contig from the Adh region 
of Drosophila melanogaster. This experiment, known as the Genome Annotation 
Assessment Project (GASP), was launched in May 1999. Twelve groups, applying 
state-of-the-art tools, contributed predictions for features including gene 
structure, protein homologies, promoter sites, and repeat elements. We evaluated 
these predictions using two standards, one based on previously unreleased 
high-quality full-length cDNA sequences and a second based on the set of 
annotations generated as part of an in-depth study of the region by a group of 
Drosophila experts. Although these standard sets only approximate the unknown 
distribution of features in this region, we believe that when taken in context 
the results of an evaluation based on them are meaningful. The results were 
presented as a tutorial at the conference on Intelligent Systems in Molecular 
Biology (ISMB-99) in August 1999. Over 95% of the coding nucleotides in the 
region were correctly identified by the majority of the gene finders, and the 
correct intron/exon structures were predicted for >40% of the genes. 
Homology-based annotation techniques recognized and associated functions with 
almost half of the genes in the region; the remainder were only identified by 
the ab initio techniques. This experiment also presents the first assessment of 
promoter prediction techniques for a significant number of genes in a large 
contiguous region. We discovered that the promoter predictors' high 
false-positive rates make their predictions difficult to use. Integrating gene 
finding and cDNA/EST alignments with promoter predictions decreases the number 
of false-positive classifications but discovers less than one-third of the 
promoters in the region. We believe that by establishing standards for 
evaluating genomic annotations and by assessing the performance of existing 
automated genome annotation tools, this experiment establishes a baseline that 
contributes to the value of ongoing large-scale annotation projects and should 
guide further research in genome informatics.