1. Summer schools
2. Treasurer's report
3. The annual conference and its relations to HLT
4. Business manager's preliminary report on HLT-NAACL 2006
5. Policies for future meetings
6. General and local arrangement chair's preliminary report on HLT-NAACL 2006

First, Andy Kehler gave a report on the scholarship competition, run by Andy and Owen Rambow, for the 2006 two-week summer school at Johns Hopkins. Of the group of 14 applicants, which included undergraduate, masters, and early PhD students, 12 were offered support; of the other two, one was an overseas candidate and the other's application packet did not indicate a clear interest in NLP. Support offers were made using a three-tier system based on the committee's estimates of the students' financial needs:

We then discussed whether NSF could fund the student fees, since it has provided student support for related events, such as this year's HLT-NAACL doctoral consortium; there seem to be NSF programs that supply $20-30K for such purposes. However, it wasn't clear whether only US citizens could receive such funds, or whether the fact that the school is associated with the NAACL would suffice. (Also, NSF funds require the use of US-flag carriers.) One suggestion was that the NAACL itself could apply to the NSF. However, others doubted that the NSF would give money directly to the NAACL for this purpose.

Next was the topic of the Linguistic Society of America 2007 summer institute (LI)'s proposal for NAACL co-sponsorship and support; the request was for $5000 to fund computational-linguistics faculty and $10000 for student support. We first reviewed the reasons behind the NAACL's $3000 package for the 2005 institute. Then, we discussed the LI 2007 course offerings, which represent a much stronger computationally-oriented set of classes and instructors than was the case for the previous institute. Another point of discussion was the comparison to the style, content, and administration of the Johns Hopkins summer-school classes. For example, the LI is having short (half-day?) intensive tutorials followed by multi-week courses. It was agreed that we needed more information about tuition and lodging costs.

(return to top of minutes)

[2] Treasurer's report

Christopher Manning presented his report (pdf file). The 2001 "split" in the graph is due to a first payment given in advance to get the 2001 meeting started.

It was noted that conference accounting has been delayed (at some point, it had been arranged for seven pounds of paper records to be transferred to Chris at this meeting). Christy Doran was thanked for still being very responsive in providing data regarding HLT-NAACL 2004, and Kathy McCoy was also thanked for being very helpful (she does half of the NAACL accounting). It was suggested that in the future, it is good to make sure data goes to the Business Manager as well, although it did not seem to be necessary to have the Business Manager actually maintain the conference budget. The ACL directly handled outgoing (for example, it gave $120K to MITRE) and incoming funds. HLT-NAACL 2004 was predicted to be (quite?) profitable, and a profit of $15K was predicted for the 2006 meeting.

We then talked about other possibilities for increasing funding, such as reducing "swag" given away at conferences or developing more opportunities for corporate sponsorship such as named events, more industry-oriented tracks (perhaps the demo track could be adapted), and recruiting opportunities (perhaps an interview or presentation room for students could be provided). A point of comparison is the Text Analytics Summit and its list of industrial sponsors.

At the current meeting, Microsoft Research was hosting an invitation-only party, and IBM held a student event, but we did not know whether the NAACL received any funds and whether there were strings attached.

One thought was to have a study group on increasing corporate involvement. Some possible targets would be statistical-analysis companies like SAS or SAP, database companies like Oracle, or big pharma. Such an initiative seemed to be something worth bringing up to the ACL Exec.

(return to top of minutes)

[3] The annual conference and its relations to HLT

We then began to consider the HLT framework with respect to future NAACL conferences. There seemed to be three main issues involved: the relationship with the so-called "funders"; outreach towards/inclusiveness of IR and speech; and institutional issues, such as the role of the HLT board.

The involvement of the funders has decreased greatly from the past; previous involvement did include providing money and doing the actual asking of people to serve in conference-organization positions (this was thought to cut down on declinations). Opinion among NAACL members seems to be mixed: some are very happy with the idea of (more) funder involvement, others are not.

The outreach/inclusiveness feature seemed good to all concerned. Speech seems better integrated, perhaps because the short-papers track resembles speech conferences, perhaps because of current research trends. The IR track might be affected by competition with SIGIR, WWW, and ICML, and recruiting for IR tutorials was less successful than desired, although the tutorial track did receive many submissions (more than 20). Since the NLP track receives the bulk of the submissions, it might make sense to have a second NLP co-chair.

We then discussed institutional issues. The conversation started with the question of whether the Advisory Board should serve an advising-only role (this was sometimes referred to as a "True Advisory Board (TAB)") or whether it should be empowered to make decisions as well. A TAB could be more inclusive: besides people from the NAACL Exec, IR and speech communities, the previous general chair, and the "traditional" funding agencies, it could include slots specifically for industry or other organizations, such as NSERC or (institutionally) NIST. But a TAB would mean another body (with separate meetings, etc.) on top of a presumed organizational committee for the conferences, and wasn't clear to all that it makes to have an advice-only entity.

It also remains to be checked that the ACL is OK with ACL meetings in North America having the three IR/NLP/Speech tracks.

This left the issue of the name, noting that for better or for worse, "HLT" has connotations for some people stemming from past history. After tossing around "Language technologies", "HLST" ("S" for "science"), "HLTS" (as more euphonious) and discussing whether a hyphen suggests a merged conference (there being no separate HLT conference currently), we settled on "NAACL HLT", which is analogous to "ACM SIGIR" (NAACL being the "brand name").

(return to top of minutes)

[4] Business manager's preliminary report on HLT-NAACL 2006

Priscilla Rasmussen reviewed the current status of the conference. She had recorded 626 pre-registrations and 15 on-sites. She estimated more than 700 registrations by conference end, and thought the location of ACL in Sydney, Australia was a factor.

Google and Microsoft contributed a substantial amount towards swag and booths.

The hotel's initial estimate of $25-26K for AV needs and $7500 for one week of internet service was reduced to $22K total after she went for outside bids. (The General Chair's policy was to provide wireless internet in an internet cafe but not in the ballrooms.)

Room rates and availability were another issue. It was not possible to get the conference rate for workshop nights, which meant a significant price increase ($160 extra per night). The arrangement with the hotel was: if 85% of the 1100 room-night block is filled, then there will be no charge for the conference meeting rooms; if 80-84% are filled, then the charge for all the conference meeting rooms will be a flat $2000 per day. Priscilla thought that we would hit the 85% mark.

(return to top of minutes)

[5] Policies for future meetings

Strong enthusiasm was expressed about the Rochester local arrangements team and the chairs who had agreed to serve so far for NAACL HLT 2007.

It is not too early to start thinking about locations for 2009. It was suggested that the initial round of competition be one just of formal expression of interest, and that after this the Business Manager should communicate with those interested to help them understand what is involved. The thinking was that needing to provide a budget in the first round is considered too intimidating. Another point for the future is that calls for bids should specify what are acceptable date ranges. For instance, the April dates for 2007 would usually conflict with EACL, although that will not be an issue this time around.

It was suggested that reviewing policies be made at the ACL level (and that it be ensured that people look at the handbook in which the policies are ensconced): for example, the PC chairs could be told, "here are the settings you should use for the START system".

Authors need to be reminded after acceptance decisions are made that they need to withdraw papers that have been submitted to multiple venues (some authors initially forgot to withdraw papers from EMNLP after receiving ACL acceptances).

It was thought that it would be nice in 2007 to have a thank-you reception for the organizers at the meeting.

(return to top of minutes)

[6] General and local arrangement chair's preliminary report on HLT-NAACL 2006

Robert C. Moore and Satoshi Sekine reviewed their experiences. These are described more fully in the General Chair's report below.

END OF MINUTES

(return to top of minutes)

****************************************************************************

HLT-NAACL 2006 General Chair's report

Robert C. Moore,
October 17, 2006

In general I thought the conference went well. Attendance was close to 700 (Pricilla, of course, has the exact totals), which I believe makes it the best attended meeting sponsored by NAACL so far. There were two major problems, both created by the conference hotel. The first problem was that the hotel refused to make any more guest rooms available at the conference rate than the minimum number they guaranteed, even though they had rooms available. What I expected, and what Priscilla confirmed is the usual practice, was that while they would not guarantee any more rooms than the contract called for, if more rooms were available, they would provide them to our attendees at the conference rate, which they refused to do. The second problem was that, while we were verbally assured that we would have exclusive use of the meeting room space on the second floor of the hotel throughout the period of the conference, the hotel in fact booked several additional events that we had to work around. While we had sufficient room for all our sessions, some of the competing events were quite noisy, which disrupted several of our sessions in adjacent space. I suppose the lesson to be learned from this is to study hotel contracts very carefully, and make sure that all our expectations are explicitly spelled out.

A major innovation in 2006 was that the student research workshop (SRW) was replaced by a doctoral consortium, during which advanced graduate students were given an opportunity to present an abbreviated "job talk" and get feedback from a panel of senior figures in the field. We decided to try this, because at the previous two HLT-NAACL conferences, the SRW received barely enough submissions to fill its sessions, allowing for almost no selectivity. At HLT-NAACL 2004, the SRW received only 12 submissions, accepting 10. The international ACL conferences, in contrast, always seem to generate much more interest in the SRW. ACL-Coling 2006, for example, received 40 submissions to the SRW, of which 15 were accepted for presentation. The switch to the doctoral consortium format at HLT-NAACL 2006 was successful in increasing submissions to 18, of which 10 were accepted. I attended most of this event. It seemed to me that almost all the students took it very seriously, and got useful feedback from the panel. The costs of the event were covered by a combination of a grant from NSF and a donation from Microsoft.

HLT-NAACL 2006 followed the practice established in 2004 of having three co-chairs for most positions, one each from NLP, IR, and Speech. We did this specifically for positions that dealt with the program (main program, tutorials, workshops, and demonstrations) and publicity. The other positions, where it did not seem as important to have the full range of disciplines represented (sponsorship and exhibits, publications, local arrangements), we had at least two co-chairs, but we did not concern ourselves with disciplinary diversity. As in 2004, we combined sponsorship and exhibits, since most exhibitors are in fact sponsors exercising the benefits of their sponsorship. As far as I can tell, the basic mechanics of organizing the main program, workshops, and tutorials went smoothly. The START conference management package seems to be working well for us, although I gather the vendor seemed somewhat overwhelmed by the extensive use we were making of it (including all the workshops), but I feel that is an issue for ACL to negotiate with him.

My impression is that the overall quality of accepted papers was quite high. The acceptance rate was 25% for long papers and 41% for short papers, which appears to be in the sweet spot where the acceptance rate is low enough that quality of the presented papers is high, but high enough that there is not too much complaining about good papers being rejected. There were 257 full paper submissions, which was many more than the 168 submitted in 2004, but much less than the 402 submitted to HLT-EMNLP 2005. Since EMNLP is normally a separate conference from HLT, perhaps it is more appropriate to compare that 402 to the total of papers submitted to HLT-NAACL and EMNLP in 2006, which was 491 (257 + 234). Viewed that way, the monotonic increase in paper submissions seems to be continuing, and we very well may see even more submissions to HLT-NAACL 2007, although the very early submission deadline may diminish that somewhat.

The distribution of papers remains heavily weighted towards NLP. I don't have the exact numbers at hand, but my impression is that there is a healthy submission level of speech papers, and that HLT is beginning to be a venue of choice for certain speech-related topics, including language modeling and spoken-dialogue issues. Getting significant numbers of IR-related submissions remains a challenge.

Handling of publications seemed to go particularly smoothly, thanks in large part to the high degree of automation that Jason Eisner developed for ACL 2005. We briefly discussed going with CD-only proceedings, as at ACL-Coling 2006, but in the end decided against it. I believe that it is only a matter of time before this becomes the norm, however.

I feel that there are two additional minor, but chronic problems that affected HLT-NAACL 2006. One was the lack of clear lines of authority, which was noted by Julia Hirschberg in her 2004 conference report. It is just not clear what the conference organizers can decide on their own, and what must be referred to the NAACL Exec, the HLT Advisory Board, or the ACL Exec. This could be helped by drafting a specific HLT-NAACL conference handbook. Trying to use the ACL conference handbook for HLT-NAACL leaves the organizers uncertain which references to the ACL Exec to interpret as meaning the NAACL Exec and which still mean the ACL Exec; and the HLT Advisory Board is nowhere mentioned.

Another problem that also seems to be chronic is appointing the conference organizers early enough. In 2006 and again in 2007 the general chair was not selected nearly early enough to come anywhere close to meeting the timetable called for in the ACL conference handbook. So, although everything did get done in 2006, much of it was at the last minute. I would urge that the NAACL Exec and HLT Advisory Board try to appoint a general chair for one conference well before the preceding conference has occurred and before the contracts have been signed for the venue of his/her conference so he/she can play an active role in that area. This means that if there is to be an HLT conference in 2008 (perhaps combined with EMNLP again), we should be working now (October 2006) to identify a general chair and a site.

(return to top of minutes)