Mark Dredze is an Assistant Research Professor at Johns Hopkins University in the Department of Computer Science in the Whiting School of Engineering. He is a member of the Human Language Technology Center of Excellence (HLTCOE) and the Center for Language and Speech Processing (CLSP). He received his PhD from the University of Pennsylvania in 2009. Mark has served as an area chair for ACL and NAACL. His research focuses on machine learning in natural language processing, with applications including information extraction, social media and health informatics.
The goal of the annual NAACL conference is to select and promote the best research. We continuously improve our paper review and selection processes to ensure a high quality conference. In recent years, other Computer Science communities have developed and tested new ideas for improving their processes. My goal is to bring the best of these ideas to NAACL.
Review Assignments: Over the past two years I have developed and run an automated system for assigning reviewers to areas (implemented at ACL 2012 and NAACL 2013.) The system has resulted in a better area load balance and fewer reviewing emergencies. I’d like to formalize this system and make it a community standard. Furthermore, I’d like to extend the idea to automating paper assignments using machine learning. The ML community (e.g. ICML and NIPS) has done this and we should do the same.
Poster sessions: While accepted NAACL papers are assigned an oral or poster presentation based on the most suitable presentation format, the reality is that posters are perceived as second tier papers. Yet each format has its own advantages; poster presenters benefit tremendously from extended informal discussions as opposed to short Q&A sessions. Other conferences (e.g. NIPS) view posters differently: every paper is presented as a poster, and some are given oral presentations as advertisements. We should consider a similar system and better utilize our own poster sessions.
Paper formats: NAACL has a tradition of soliciting short papers, which enables excellent contributions that don’t need 8 pages. However, our guidelines for short papers (e.g. “work in progress”, “a negative result”) don’t reflect our expectations. Additionally, papers are forced into either short (4 pages) or long paper models (8), but some papers need only five pages and as a result are penalized by reviewers. Other communities have used more flexible systems; papers can vary in length as needed (up to 8 pages.) Papers are judged based on the quality of contribution for the length of presentation, preventing paper padding to reach a higher limit (or cut unnecessarily to reach 4 pages.) We need to reconsider the goals of the short paper category and consider these other models.
Program Chair Tenures: Each year we have (typically) two NAACL program chairs who, after spending a year learning how to run the conference, end up with a list of great ideas for improvements. However, its a one year job, so the list is never used and we lose valuable ideas. One alternative would be two year PC tenures with a new PC member each year. The junior member runs the typical proceedings process, while the senior member focuses on conference changes based on previous experience. This structural change would ensure an ongoing effort to improve NAACL.
I intend to draw upon lessons learned by other CS communities to bring the best ideas to NAACL.