NAACL 2013 final report

by Lucy Vanderwende, Hal Daume and Katrin Kirchhoff

TACL

Co-located Conferences

NAACL, main conference:

Recruiting reviewers

Areas for review

New review form

Accept/reject by Area and Length

Extra page after acceptance?

Best paper

Scheduling concerns:

Previous publications:

Videorecording

NLP-idol:

Workshops on Tutorials Day:

One-minute madness of posters:

Feedback from the NAACL organizing committee:

tutorials: Katrin Erk and Jimmy Lin

demonstrations: Chris Dyer and Derrick Higgins

student research workshop: Annie Louis and Richard Socher

workshops: Sujith Ravi and Luke Zettlemoyer

publications: Matt Post and Colin Cherry

Webmaster:

Student Volunteers: Jacob Eisenstein

Sponsorship

Publicity

TACL

NAACL 2013 was the first conference to include papers accepted at TACL. It was agreed with the TACL editors that we would offer the TACL papers oral/poster presentation slots in roughly the same split as the NAACL oral/posters, which happened to be 2/3 and 1/3 this year.  We also decided to ask TACL for the number of papers to be presented at TACL only after our final NAACL decisions were made in order that it be really clear that NAACL decisions were completely independent of the TACL papers. Only 6 TACL papers asked for presentation slots at NAACL.

As it turns out, TACL paper authors were not aware that they might be given a poster slot and this realization (and expressed discontent) led to much discussion on the side of the TACL organization, regarding the original understanding that TACL papers would be given either oral or poster slots. Our initial allotment had 4/6 TACL papers as orals; we were able to add one more. We regret that there was only one TACL paper was the only poster, but putting it as an oral paper would have necessitated an additional parallel session or an uneven number of oral presentations in sessions that are already long.  Had we known that all TACL papers "should" be oral, we would have organized the oral presentations differently (one possibility would be to present all TACL papers as a  separate parallel session; however, we expressly decided that it was better both for TACL and for the conference to arrange TACL papers topically in the program, something ACL 2013 seems to have done differently).

As mentioned below, we are dismayed that there is a continued perception that posters are somehow “less” than oral presentations.  This concern can be addressed in several radical ways:  TACL papers can be presented as 15-minute plenary talks – short, but broad impact; all NAACL papers can be given as short talks with no poster session - there may be quite a bit of support for this in fact, anecdotally; all NAACL papers can be posters, with few plenary sessions; etc etc. A real question is: as more and more TACL papers are accepted, will the number of conference papers remain constant or will it go down commensurately?  will TACL papers ultimately have an affect on the acceptance rates of the main conferences?

To avoid future issues, the community might want to come to an agreement about oral/poster distinctions for TACL papers. For instance, if a TACL paper is offered a poster slot at NAACL, can they "bargain up" to an oral slot at ACL or EMNLP? This seems like a bad path to go down. It might also be worth considering that current TACL papers "look like" normal (NA)ACL/EMNLP papers, but many journal papers look substantially different and are in fact less well suited to oral presentation than many "normal" conference papers. Making hard decisions before this plays out also seems challenging.

finally: there should be a separate START track for TACL papers where TACL authors upload all the relevant information, so that the program creation is simplified and easier to update/maintain

Co-located Conferences

*SEM/SEMEVAL

NAACL is not involved with the organization of *SEM, which aims to be an independent conference, however there are several points at which they meet: the start system, local arrangements, publications.  *SEM means significant additional work for the NAACL pubs chairs, which really should be recognized and acknowledged.  If the co-location continues, then perhaps there ought to be joint pubs-chairs to share tasks.

Further, there were significant scheduling concerns for SEMEVAL, whose status appears to be unclear and which should be clarified and codified going forward.  As discussed in the board meeting, no co-located conference should have its own workshops, without informing the main conference’ workshop chairs.

NAACL, main conference:

Recruiting reviewers

if you want to continue this approach, contact Mark Dredze

Both of the Program Chairs and myself dislike immensely the “race for reviewers”.  We also felt that we wanted to encourage diversity in the reviewing pool, as we had heard from numerous potential reviewers that they would like to review in a specific area but are never asked. With this in mind, we implemented the following: we asked area chairs for candidate reviewers. We then took the union of the candidate lists for all areas and asked these reviewers to create their START account (this is now an unnecessary step as the pool of possible reviewers now is already enlisted in START) and to select which areas they would like to review for. We are indebted to Mark Dredze for pioneering this idea. Then, when we knew how many papers were submitted in each area, we used Mark’s allocation program to find a good distribution. This ensured that the reviewers who were recruited had papers assigned to them, but not too many, and there is no communication necessary with two or more area chairs to keep the review load manageable. In addition, no area had either surplus reviewers or too few. It is the case that area chairs then were relying on the input of people they knew less well in some cases, but anecdotally, the area chairs appreciated the input of people they may not have asked to begin with, which does indeed increase diversity in reviewing, our original goal.  Keeping track of member’s preference to review for specific tracks could be part of the START profile annually updated. If we do begin to rely on the existing pool of reviewers to re-recruit, then we should be very careful to continue adding new candidate reviewers each year.

Areas for review

Here is a list of the areas that we used in NAACL2013.  While fairly conventional, there are a few points worth noting.

  1. separate area chairs for generation and summarization (traditionally, lumped together though no one really knows why)
  2. we renamed “machine learning” to “machine learning for Language Processing”
  3. additions, which have always been checkboxes, but had not had a designated area chair previously
  1. language resources and evaluation
  2. linguistic aspects of CL
  3. NLP-enabled technology

Discourse Dialogue Pragmatics

Document Categorization & Topic Clustering

Generation

Information Extraction

Information Retrieval and Question Answering

Language Resources and Evaluation

Linguistic Aspects of CL

ML for Language Processing

Machine Translation

Multimodal NLP

NLP-enabled Technology

Phonology, Morphology and Word Segmentation

Semantics

Sentiment Analysis and Opinion Mining

Social Media Analysis

Spoken Language Processing

Summarization

Syntax, Tagging Chunking and Parsing

Additionally, we drastically reduced the number of "checkboxes" for keywords. For instance, in the previous version, there were at least 10 checkboxes that had to do with machine learning (both synonyms as well as hyponyms).

New review form

We made some changes to the review to attempt to (a) make it more clear and (b) push reviewers in the directions we deemed important for decisions. The new review form is available at http://naacl2013.naacl.org/naacl-hlt2013_reviewform.html and a list of the changes is at http://naacl2013.naacl.org/Differences.aspx, but copied below for completeness:

First, the questions have been re-ordered to first call attention to the most important aspects of a paper. The remaining changes are in hopes of making the review options more clear, and giving them more focus. The changes we feel are most important are listed first:

  1. Overall Recommendation: Now focuses on whether a reader is likely to learn something from this paper. The top two options for ratings are new and more specific than before.
  2. Appropriateness: the scope has been expanded to also welcome papers that address more linguistic aspects of CL.
  3. Thoroughness: Now has an opportunity to reviewers to give high scores to papers that show not just when their approach works, but also when it does not.
  4. Originality/Innovativeness: This has been expanded to highlight originality not just in approach, but also in problem scope. We also added a note that a low score here should be justified with direct literature pointers in the review body.
  5. Clarity: The focus here is on understandability: could the reader actually figure out what was done and why it was done.
  6. Meaningful Comparison: The new focus is on whether the authors do a good job of situating their work The description of the options is completely new to focus on comparison, not bibliography.
  7. Impact of Ideas or Results: Replaced notion of whether this work will be cited with whether it will be read and used.
  8. Presentation Format: This is no longer shown to the authors, and reviewers are given the option of selecting "No Opinion"

Accept/reject by Area and Length

The official, final statistics for the conference are below. These statistics exclude four papers that were withdrawn prior to review. It also excludes 27 papers that were rejected without review (for a variety of reasons). However, it includes seven papers that were withdrawn after review (most of these had quite low reviewer scores) and treats these as "not accepted." Any previously released numbers that are not in line with those below are due to differences in how these "strange" paper cases were counted.

                                                     

Long

Short

Total

Track

Sub

Acc

%

Sub

Acc

%

Sub

Acc

%

Discourse Dialogue Pragmatics

18

4

22.2%

6

2

33.3%

24

6

25%

Document Cat. & Topic Clustering

21

4

19%

14

4

28.5%

35

8

22.8%

Generation

3

1

33.3%

1

0

0%

4

1

25%

Information Extraction

20

6

30%

12

6

50%

32

12

37.5%

Information Retrieval and QA

12

3

25%

6

1

16.6%

18

4

22.2%

Lang Resources and Evaluation

10

4

40%

7

2

28.5%

17

6

35.2%

Linguistic Aspects of CL

10

6

60%

7

2

28.5%

17

8

47%

ML for Language Processing

25

9

36%

4

2

50%

29

11

37.9%

Machine Translation

55

16

29%

29

9

31%

84

25

29.7%

Multimodal NLP

5

2

40%

0

0

0

5

2

40%

NLP-enabled Technology

9

1

11.1%

3

0

0%

12

1

8.3%

Phonology/Morph/Segmentation

9

2

22.2%

8

1

12.5%

17

3

17.6%

Semantics

36

8

22.2%

18

6

33.3%

54

14

25.9%

Sentiment Analysis and Opinion

10

2

20%

7

3

42.8%

17

5

29.4%

Social Media Analysis

15

6

40%

6

1

16.6%

21

7

33.3%

Spoken Language Processing

6

5

83.3%

15

9

60%

21

14

66.6%

Summarization

9

4

44.4%

2

1

50%

11

5

45.4%

Syntax Tag Chunking and Parsing

21

6

28.5%

17

2

11.7%

38

8

21%

overall

294

89

30.2%

162

51

31.4%

456

140

30.7%

As expected, there is high variance both in terms of the number of submission (from 4 for generation to 84 for machine translation) and the acceptance rates (only 8.3% for NLP-enabled technology and as high as 2/3 for spoken language processing).

Extra page after acceptance?

We did not give the authors advance notice that the final length of their paper could be up to 9 pages, with additional pages for references. Not sure whether that makes a difference to how the paper is written for submission, but it would be nice for authors to know in advance.

Best paper

Does NAACL always have best papers is all the categories, best long, best short and best student? We have asked that the NAACL committee come to a policy decision on this. In 2013, the best paper committee elected not to choose a best long paper. Several people were quite disappointed at this decision.

Prior to NAACL 2013, The NAACL board did make a policy decision that a small number of runners-up could be made public, when there is such a list (note that the runners-up is not the same as the candidate best papers when there are more than a few candidates.) Again, for NAACL 2013, the best paper committee did not select any runners-up.

In NAACL 2013, we reviewed the best papers based on camera-ready copy as that will be the form of the paper going forward and gives the authors the opportunity to present their best work having incorporated the suggestions from reviewers. HOWEVER, reviewing camera-ready meant that reviewing was not blind, and creating the program schedule was much delayed, which causes problems for registration (the program schedule should be available when people register and make travel arrangements.) Daniel Marcu’s comment that a nominated paper is likely to be mature work and so probably wouldn’t undergo many modifications is probably right by and large, so that would suggest to return to a the practic of reviewing for best paper based on submitted, not camera-ready, copy.

Scheduling concerns:

Acceptances need to be published early enough for people to apply for US visas. This will be an issue for ACL 2014. Check with Priscilla about the timing for visa!!!!

The program schedule needs to be published before registration opens so that people can book their flights. (This is what we didn’t get quite right for NAACL2013 - the PC chairs don’t see the complaints but Priscilla does.)  Making the program schedule earlier also means fewer requests for papers to be moved around, and gives you more time to find session chairs too and include those in the published handbooks.

Request applications for student volunteers as soon as paper acceptances go out. They know they need to attend and if they need additional funding. Get student volunteer chair to work on that right away (I started too late on that and it caused a delay on getting everyone scheduled, so Priscilla didn’t get a chance to look over the schedule and make sure everyone was covered. Just trying to reduce last minute scrambles is better over all …)

Previous publications:

There should be a reconsideration of what “previous publication” means and what is/is not allowed.  For example, we did not allow a paper that had been uploaded to a personal website (this was made clear on the "sample LaTeX document" but not on the CFP), or submitted as an ArXiV paper, as these violated the goals of anonymity.  We have asked NAACL exec for a policy on that, but you may still face this question.

Videorecording

If there is a decision to make videorecording a permanent addition to NAACL conferences, we recommend finding a way to include the recording of the posters as well.  It is stated policy that there is no quality difference between papers and posters, so we should offer the same chance for the authors of posters to be recorded as well. One possible solution would be to have a booth with recording self-service, where a poster presenter can go and record a 5-minute presentation with their poster in the frame. Another would be to allow presenters to upload a self-recording to the same location as all the oral presentations.

NLP-idol:

in 2012, the program committee held a plenary event NLP-Idol, which was considered a great success.  We could not follow up with anything similar this year, but it is probably worth re-considering, perhaps by thinking of how to establish a best paper with 10 years retrospective and with 5 years retrospective, with/without a discussant. I believe that it would be not too hard to find sponsorship for such a paper category and would provide at least in part what was attractive about the NLP idol, i.e., to bring back some well-known paper that had significant impact on the field (but may not have been recognized as such in its first publication).

In fact, such a best paper award might make a very nice closing session, as the award winner could be kept secret until the close of the conference (to everyone but the winner, naturally). Lucy personally would be sad for that year’s best papers to be part of closing ceremony as it doesn’t leave time for those authors to talk to interested fellow researchers throughout the conference - putting those papers on Tuesday allows maximum impact.

Workshops on Tutorials Day:

In 2012, the NAACL exec did decide to allow 1-day workshops to be held on the same day as tutorials. This would be particularly useful when there are many workshops and/or not so many tutorials. As mentioned below, the tutorials chairs needed to solicit proposals and we realized at too late a stage that perhaps there weren’t enough tutorials to attract people to attend the pre-conference day. At that point, the workshops had already published their dates and it couldn’t easily be changed. So I would advise: get the tutorials set up early, so that adding workshops could still be a possibility.

One-minute madness of posters:

Not sure whether ACL does this, but if you do, ask Joel Tetreault once again :-) he happily volunteers and does an awesome job

Feedback from the NAACL organizing committee:

tutorials: Katrin Erk and Jimmy Lin

 

* It was good that there were two co-chairs. Although running the tutorial isn't an overwhelming load, it's nice to have backup.

 

* We had asked tutorial proposers to first submit a proposal, and then, after acceptance, to submit a tutorial abstract and speaker bio.

This turned out to be unnecessary in almost all cases as the revised and accepted proposals were already in the abstract + speaker bio format. In the future, we would suggest making sure that the proposals are in the right format, and ditch the requirement for submitting a separate abstract.

 

* This year, tutorials were accompanied by an extended abstract in the proceedings. We think this is a good idea, as it is a formal record of the tutorial having taken place. However, this time there was a bit of a mixup and we only realized late that this had to be done, and did the formatting ourselves as there was little time. Next time, the authors should be given the opportunity to prepare that document themselves.

 

* Some of the tutorials we solicited were tutorials that were particularly popular in the previous year. This worked very well, as presenters already had the proposals ready to go, plus those tutorials tended to be particularly popular again this year. We would suggest taking this strategy from the beginning, to solicit those tutorials early.

 

* Leave enough time between tutorial proposal submission deadline and the time when tutorials need to be finalized. We had cases where tutorials needed much work to be acceptable, including finding a co-presenter, so it was important to have enough time for this to happen.

 

from the NAACL committee:

no hardcopy for slides offered to participants, but should establish a point where presenters can upload their slides in advance for immediate download by the participants

demonstrations: Chris Dyer and Derrick Higgins

Task/Milestone

Date

Get informal commitments to serve on review committee for demonstrations.  (We had 17 submissions in 2013, so a committee of 30 allowed us to have each proposal reviewed by 3 people with each reviewer having a max of 2 assignments.)

Get set up as administrator in Softconf system.  If you haven’t used Softconf in this mode before, you may want to set aside some time to explore the workflow for conference arrangements and set parameters appropriately.  For example, for demonstration reviews, we chose not to enable blind reviewing, since demonstration proposals can be almost impossible to anonymize.

Post Call for Demonstration Proposals to ACL Member Portal.  Priscilla Rasmussen helped us to do this step.

Fall, 2012

Load reviewer info into Softconf

Send out link to reviewers (through Softconf)

Design review questionnaire/form in Softconf

Post Second Call for Demonstration Proposals to ACL Member Portal.

January, 2013

Submission Deadline

February 9, 2013

Assignment of reviewers to submissions

Monitor reviews and send reminders

Decide on accepts/rejects

Send out notification of acceptance/rejection

March 29, 2013

Send out request to submit camera-ready versions of demonstration papers

Camera-ready deadline

April 26, 2013

Post list of accepted demonstrations to NAACL/HLT website.  The organizing committee relayed our list to the person in charge of the web presence.

Assemble conference “book”, following instructions provided by publications chair(s)

May 5, 2013

Send note to demonstration authors with details of local setup: space and equipment provided, date/time of demo, etc.  Priscilla Rasmussen was very helpful in pulling this information together for us this year.

Conference

June 9-14, 2013

student research workshop: Annie Louis and Richard Socher

Overall, the organization was smooth and the chairs were very helpful. Especially during the book compilation stage we had many questions and were able to solve them easily and quickly with the publication chairs.

 

I had one suggestion, that new emails, forms (copyright, submission pages) created on START could be archived. That would be helpful for next years organizers.

 

Also a suggestion for the START system.

- It has provisions for requiring other documents from an author together with the paper. For example for the SRW, we also obtained a CV and letter from advisor for each student. However, when setting up for reviewing it was not possible to only show the paper to reviewers and hide the CV/letter. Both show up or none. It would have been helpful to have some more options for setting this up.

From NAACL committee: since not run through workshops chairs, there is a chance for things slipping through the cracks. It may be better to have the workshop chairs include the SRW in their list of workshops, even though not selected by them.

workshops: Sujith Ravi and Luke Zettlemoyer

The wiki (http://aclweb.org/adminwiki/index.php?title=Workshop_chair_duties) and intro email from the organizers were great, everything went smoothly according to plan.

 

The joint reviewing with ACL and EMNLP was good overall. There was a strong preference for ACL, mostly based on location I think (Sofia > Atlanta). NAACL really benefited. The coordination with EMNLP was a bit more challenging, since the conference is later and was (at least this year) pretty last minute with everything.

 

Two small recommendations. I would make it very clear in the joint call that venue assignments are final. We had a few complaints and requests to switch to ACL. Also, be careful to check for conflicts between workshops before assigning them to days at the conference. Some organizers have multiple workshops, etc. Neither of these issues were a huge problem for us, but both required a little extra work.

And NAACL committee would add: It’s important to coordinate recommended submission dates, camera-ready dates with the pubs chairs. Also, to try to get the submission dates published early enough so that people can plan more than 2 months in advance to submit a workshop paper. Ideally, the workshop submission date is later than acceptances for ACL/NAACL.

publications: Matt Post and Colin Cherry

Here are my thoughts on how to improve the publications chair job, per your request. I've CC'd Colin in case he has anything he'd like to add or contradict.

 

1. The Publication Chair job should always be done by two people serving overlapping two-year terms.

 

Colin and I worked really well together, I think, and I don't know how either of us would have done all the work alone, especially with the new responsibilities we took on this year (the printed and digital handbooks). Now that I've done it, I know exactly what I'd do next time, and don't fear it at all. I think the process each year is needlessly difficult because of all the knowledge that is lost from year to year in bringing a new person on board. I think this year we also saw that a knowledge pub chair can really smooth over a lot of the organizational process, providing help and support to the general and PC chairs. And had we known what we know now, earlier pieces could have gone more smoothly.

 

Another asset to two-year pub chair terms would be that each chair would have a real incentive to improve the process. I am very impressed at all the work that previous chairs have done, but I think what happens is that most researchers serving this position are exhausted at this point, and just want to get back to research. If serving a two-year term, they would have the incentive (and therefore the justification to take even more time out of their schedule, post-conference) to take steps to improve the process for the following year.

 

2. I would recommend organizing a short, ten minute phone call with an organizer of each workshop, soon after they are accepted. On the agenda would be:

 

- Camera-ready deadlines and the book deadline

- Explaining the process to new organizers who are unfamiliar with START

- The process (and importance) of ensuring that paper metadata is correct for all of their papers, which is important for the ACL Anthology and for the handbooks' indexes.

 

The reason for the phone call is that these are important, and that is too easy to get lost in the flurry of emails that we all deal with every day. A phone call would make sure that everyone is on the same page and would make the information more memorable.

 

3. It really helped maintaining a web page with information both for authors and workshop organizers. We'll keep this up for some time: http://cherryc.github.io/naacl-pubs/

 

4. Our biggest issues were with *SEM and SemEval: They did not stick to our recommended schedule, and had a ton of last-minute changes and requests that trickled in weeks after the event. The incentive structure here is wrong: as a co-located conference, they are independent, yet we had responsibility for both their proceedings and handbook integration. I'm not sure how to handle this in the future, but maybe there should just be a single pub chair for co-located conferences. Doing that phone call early on may have helped, as well.

 

5. START: The biggest issue I ran into was workshop chairs not making the "order" files in START machine-readable. That caused huge headaches for the scripts I was writing to auto-generate the printed and digital handbooks. I had to spend a lot of time hand-correcting those files, and then every time they changed with last-minute updates, I had to do it again. Next time, I would make the instructions for this file much more explicit, and it would also be nice if we could put a validator inside START.

 

I actually have a lot of thoughts about how to improve START for ease of future use. There are a number of other issues that came up that are not allowed in the current ACLPUB package, and it would be really nice to have a small budget to hire Softconf to implement them. A discussion has already started with Rich Gerber about what we'd like to do.

 

6. I think it would be good to consider moving to a one-column paper format, for online / digital device readability. COLING did this with some success, and if we wanted to do it, now would be a good time to get the details worked out.

 

7. This year, we introduced the idea of enforcing B&W readability, and decided it should be the pub-chairs' job to enforce it. This was a ton of work -- too much. Colin came up with a better idea: convert  papers to B&W (or grayscale) at submission time, and give those versions to reviewers. That way if a graph or figure is unreadable, the reviewer will naturally catch it.

 

That's all I can think of for the moment. I hope that's helpful! It's been a pleasure working with you and I hope we have the chance to again!

 

Sincerely,

Matt

I agree with everything Matt has said.

 

A lot of what Matt has covered is kind of a general strategy for improving the PubChair position. Most of my notes are instructions and advice to future pubchairs that will help them do the job assuming it stays mostly the same. Matt and I have been maintaining a shared document on any issues as they come up (a surprising portion of it has been invalidated as the position changed throughout our brief term as pubchairs, for example, all the stuff about sending USB keys to the printer will have to go). I'll past the most important of my sections below.

 

-- Colin

 

Heed the advice in the “How to Pubchair” online document (http://stp.lingfil.uu.se/~nivre/how-to-pub.html) regarding the tutorials book. Contact the tutorial chairs early and make sure they know what you need from them. The tutorials are very easy to forget about, as they have no papers, and the tutorial chairs may not even realize that they are expected to assemble of book of tutorial abstracts.

 

START issues

 

You’ll want to make sure that the workshops are seeded with the correct Final Submission page prior to camera-ready submissions. There is no reason not to do this right away as each workshop’s START page is created - the workshop chairs will expect the Final Submission system to just work without their intervention.

 

Request pubchair access to everyone’s START system (main conference, workshops, demos, SRW, tutorials) as soon as possible. Being able to check in on the workshops is invaluable in debugging their problems and in making sure they’re all on track.

 

Getting all books to have uniform wording in the proceedings stamps and in the various headers for the cdrom directories can be a pain. You can seed these manually for your book chairs, but it is a lot of copying and pasting (index.html.head, authors.html.head, program.html.head all need ACL 2010 information replaced with your conference; citation stamp should be seeded with a uniform version of the conference address, you may also want to consider seeding the CDROM tab, which can be a pain otherwise). It may be worth getting this all set up for the main conference before workshops are selected, and then asking softconf to clone each workshop from the main conference (this is just a hypothetical, but it would save a lot of work).

 

Making the Proceedings Image (for cdrom, USB key or download):

This is the one thing that does not happen inside START. You will need to run ACLPUB on a local computer to get this up and running. Do NOT download ACLPUB from its official ftp site. These instructions will tell you how to get an up-to-date version from softconf.

 

                   • Edit mainindex.html.head to describe your conference. Replace each workshop in the provided template with your own workshops, using the abbreviations included in the workshops’ CDROM tab for directory names.

                   • Download the ACLPUB package from softconf by using the the All->[zip] link under Generate. This will grab your proceedings directory along with aclpub's templates, make and bin. Move templaes, make and bin into an aclpub directory, and set the environment variable ACLPUB to point to that directory.

                   • Make a directory called conference, you will build your cdrom from here

                   • Copy make/Makefile_pubchair into conference, rename it to Makefile

                   • Make a directory called books in conference

                   • Download the tar ball for the conference and each associated event from softconf into books - give the tar balls different names or they'll overwrite one another.

                   • Untar each book, one at a time. Rename the produced proceedings directory to the abbreviation of that book as found in proceedings/meta . Make sure no one’s abbreviation uses a sketchy special character that would make for an unusual file name (*SEM should have the abbreviation StarSem)

                   • Edit conference/Makefile to replace any mentions of ACL2005 with your own conference. Edit conference/Makefile and each book’s internal makefile to comment out cdrom/ps (unless you want postscript files). The script below assumes you fixed Makefile_bookchair once, which can then over-write each book’s Makefile. You may also need to strip out any occurrence of “--no-run-if-empty” if running on a Mac.

                   • Go back to conference, run "make cdrom", note that you will have to have edited mainindex.html.head to use the abbreviation directory names listed above.

 

Here is a bash script that does steps 6 and 7, as you may need to repeat them as problems are found with books:

 

#!/bin/bash

#

# Command line arguments are book directory names used on START # To be run from within the book directory # papers srw tutorials demos BEA8 SSST-7 Meta4NLP2013 MWE2013 LASM2013 EVENTS WVL13 CLfL NLP4ITA2013 WASSA2013 # echo "Using ACLPUB=$ACLPUB"

while test $# -gt 0; do

        conf=$1;

        echo $conf

        wget -O NAACL2013-$conf-proceedings.tgz --no-check-certificate https://www.softconf.com/naacl2013/$conf/manager/aclpub/proceedings.tgz

        tar -xzf NAACL2013-$conf-proceedings.tgz

        cp -f $ACLPUB/make/Makefile_bookchair proceedings/Makefile

        abbrev=`perl -ne 'print if s/abbrev\s+//' proceedings/meta`;

        if [[ -e $abbrev ]]; then rm -rf $abbrev; fi

        mv -f proceedings $abbrev

    mv -f NAACL2013-$conf-proceedings.tgz ../tar

        shift

done

From the NAACL committee:

it would be nice for the app to be html5 for all platforms ...

Webmaster:

Something to be done on the early side. MSR funded a contractor to do the webmastering, but traditionally, it’s been handled by a student volunteer. We are happy to give you the framework that we used for the website - it was very easy to navigate, which people appreciated.

You will also do need a Logo for the conference – you may want to do that sooner rather than later as it makes for nicer website design. Priscilla has asked Lucy Roark to design a logo in the past – check with Priscilla.

 

Student Volunteers: Jacob Eisenstein

Selecting Student Volunteers was hard. We had more applicants with

full papers than we could accommodate.  I tried to figure out where

the money would make the biggest impact, and only funded one student

per advisor and per paper. When people said that they had no other

sources of funding to attend the conference, and were traveling

internationally, I turned them down on the theory that they couldn't

really attend. But in the end, I think I accepted these people from

the waitlist, because the initial estimate for the number of

volunteers was too low. Another criterion that would have been helpful

was the volunteering availability that people listed, to avoid people

who would only be available for one or two days. I took a few people

from local universities even if they didn't have papers. I wasn't sure

if this was fair at first, but it turned out to be a good idea,

because local students were able to cover volunteer slots at the

beginning and end of the conference that visitors were unable to do.

So I'd recommend that you have at least five local SVs.

 

We only reimbursed SVs for their main conference registration, not

for tutorials or workshops. We had a very small amount of money ($1000)

for travel fellowships for people from developing countries, and funded two

people this way. I think we could have been clearer about how limited this

funding was; several people only applied for this and not for the SV position.

The student research workshop also funded some students, and there wasn't

really any coordination between us. I don't think it was a problem, though.

 

Preparing the schedule was difficult. I used an Integer Linear

Programming solver for this. I can share the code, but I can't provide

any support. In retrospect, I should have asked the SVs again for

their availabilities before making the schedule. For many SVs, their

availability was different from what they had said on the original

application, forcing me to do a lot of annoying rescheduling.

Sponsorship

from Lucy:

This is more important than you think …. and a little confusing.  

The big accounts (“big 5”) are handled by Priscilla entirely (IBM, Google, Microsoft, SDL, and ?). Then there are the ACL-NAACL exec appointed two sponsorship co-chairs with overlapping terms - as GC, I never had any contact with them as they report to Priscilla.

NAACL does appoint one local sponsorship chair whose duty it is to get creative and try to find new sponsors that are perhaps more locally relevant (and not already on the existing list of contacts that the other two co-chairs are supposed to go through). But I think the communication with Priscilla is much less tight and perhaps as GC, I should have asked for progress reports along the way.

Generally speaking, sponsorship is down for NAACL. Sponsorship allows NAACL to hold the registration fees relatively low, while having nice venues and food during break and opening and poster night food.  I think that as GC, you need to be a little more involved in following up with whoever is appointed for local sponsorship (and perhaps check in with Priscilla more regularly as well regarding sponsorship). There should be plenty of institutions in the Baltimore area who could be sponsors. Of note: at least some companies have an annual budget and so should be asked early, in case they need to request money for their budget.

Publicity

We had postcards printed to distribute at related conferences during October and November, to advertise the upcoming paper deadline for NAACL 2013. This was not expensive, covered by Microsoft, and was a nice way to raise awareness. For ACL 2014, you may consider doing this at EMNLP 2013.

It is important to publicize the paper deadline and the workshops deadlines on the ACL members portal - contact Priscilla, who also has a number of other mailing lists to which the information can go out.

We used Twitter extensively to communicate, which also reaches non-Twitter users by means of a window on the main conference webpage, for high visibility.  

Prior to the conference, a number of newsletters should go out.  The first one should be in time for non-US attendees to be reminded of the need to apply for their US visas.  Then one per month is advisable with updates regarding hotels, registration deadlines, early/regular/late registration dates, etc.