Minutes for 2019

Slides from NAACL 2019

NAACL 2019 Business Meeting (Julia Hockenmaier)
NAACL 2019 Treasurer Report Presentation (Jonathan May)

Minutes from NAACL Board Meeting

Date: June 3, 2019
Location: NAACL-HLT 2019 in Minneapolis, MN, USA

Attendance

Julia Hockenmaier (Chair)
Emily Bender (Past Chair)
David Yarowsky (ACL Representative)
Jonathan May (Treasurer)
Colin Cherry (Secretary)
Marie-Catherine de Marneffe (Board Member)
Marilyn Walker (Board Member)
Luke Zetttlemoyer (Board Member)
Graham Neubig (Board Member)
Joyce Chai (Board Member)
Priscilla Rasmussen (ACL Business Manager)

ACL Anti-harassment policy and Professional Conduct Committee (Emily)

AHP started with NAACL, adopted for ACL 2014 in Baltimore.
Professional Conduct committee built to handle the complaints raised by the AHP.
Roughly 10 members on committee, trained in mediation.
People on committee serve renewable three-year terms.
Jon: are we going to change the instructions for how to handle AHP complaints.
- A: People come to the exec, exec directs them to the professional conduct committee.
David: how many complaints?
- A: not many over all.
David: how is it enforced?
A: stick very close to the AHP for formal sanctions, needs to be on a conference site. Might still offer to do mediation, even if we’re not a position to do something formal.
Jon: social activities associated with an event are also under the AHP?
- A: Yes. Even social events.
Expect annual calls for participation.
Working on an enhanced conflict of interest policy (for publications).
Trying to get softconf to maintain a list of hardcoded COIs
Marie: are we changing who you should talk to for a AHP complaint?
- A: Link with names of committee members are on the (AHP page)[https://aclweb.org/adminwiki/index.php?title=Anti-Harassment_Policy].

Election Report (Colin)

We had 191 people voting out of 1649: 11% voter turnout.
Substantial drop from last year’s 17%
- (which was a drop from the previous year’s 21%)
- reflects a slight drop in absolute numbers from last year (213 voters), combined with another 30% increase in membership.
- Need to send more reminders next year
Got started early this year - already contacted the head of next year’s nominating committee: Matt Post
Also mentioned the election software - going to use ElectAssist and not move to the cloud just yet

JSALT Summer School Scholarship (Colin)

Funds drawn: $5k for 3 students
24 applicants
- (surprising number of applications from English-as-a-second-Language-Learners)
Selected 3 students: Brazil, Peru and St. Louis
Distributed travel funds unevenly to accommodate some who had expenses well over $500
Jon: Concerned about whether or not we are giving scholarships to the same schools as ERF
- Same university in Peru, different university in Brazil.
TODO: Raise issue of whether we place a restriction on where you will be attending university next year

Emerging Regions Fund (Marie)

10 submissions (on par with previous years)
Criteria for selection:
- Number of people impacted by award
- Geography
- Novelty
We selected 3 recipients (similar to last year)
Marie raised issue of increasing funding for ERF.
- Last year voted to increase to $4,500. If the budget is better next year, can we go back to $4,500 for ERF?
  - TODO: Issue for vote, and to return to discussion
- Julia: do we get reports on impact of ERF? JSALT?
  - A: Yes for ERF.
  - A: No for JSALT.
Example: $1000: 2-3 people to Lima plus corpus construction.
Details in slides.
ERF money goes quite far.
Lyn: What about who we turn down?
- Typically turn down individual travel, or to fly down experts.
Some examples given, mostly spreading money to new sites.

Videos of NAACL Talks (Julia)

2-3 years where the videos had gone missing.
Managed to get ahold of WEYOND - where the videos were stored had gone out of business.
Matt Post working on getting videos moved to a new home.
Still under contract with WEYOND.

Diversity and Inclusion report (Priscilla)

Lots of ideas clawed back due to cost; cut back on captioning.
- Microsoft (Will Lewis) is now doing captioning for free; seems to be working.
- MS is piggybacking on top of WEYOND for the video part.
D&I efforts will be much harder to implement at venues outside North America
- D&I is very space-intensive.
  - Quiet room, prayer room, nursing room, child care room.
  - Ballroom for mentoring. -Mentoring breakfast + Birds of a Feather in this room.
    - Mentoring breakfast was very popular, as was mentoring in general.
- In every other country, you have to pay for every room (at least for conference hotels).
There are still more ideas that the D&I chairs would like to introduce.
Priscilla’s pick for least valuable rooms:
- Prayer room (only a couple of people).
- Nursing room not used yet.
- (May need to find a better way to track use of rooms)
Childcare:
- For EMNLP they did awards to reimburse childcare costs.
- This year, we provided a childcare room with toys and books, etc.
  - Also stickers (cheap).
  - Box at registration table with toys and games
  - Kids get to keep toys (avoids storing and shipping).
  - Expecting after conference:
    - Reimbursing travel awardees will be the bulk of the costs.
  - There are some childcare grants - $400 per applicant to a maximum for $4,000
  - Priscilla thinks that childcare policy is very good.
Julia: BIG is concerned that D&I initiative has grown so large.
Emily: Shouldn’t focus on how many people are impacted (re: Prayer/Nursing room, general consensus on this)
Emily: If we have a deaf participant that wants sign interpreting, then you do sign interpreting, don’t you?
Priscilla: described existing volunteer effort to support a blind attendee, how we’ve supported deaf attendees in the past.
- We have thus far provided services as needed. But D&I makes it more clear that we do this.
Emily: nursing room is very important for a nursing mother.
- Marie: nursing room could be a hotel room - Priscilla says that could work.
Priscilla: contracted for assistance for site and hearing impaired attendees, but was able to get out of it when no attendees required assistance.
Marie: post-conference survey, ask people what they found valuable and what they used?
Registration asked if you were planning to bring your child (yes, maybe, no). We followed up with everyone.
Priscilla: gender-neutral bathrooms and pronouns are easy. Stickers were also cheap and popular. Nickname on badge.
Julia: D&I also raised some money, right?
- A: Yes
- Spent on travel grants. Also reimbursing for toys and stickers.
Julia: what constraints are on conference organizers on what they can spend money on?
- A: for money that they raised, they can spend at will and get reimbursed.
Julia: cushion of money for D&I?
- A: Funds to support D&I drawn from conference budget.
Jon: should this decision have gone through the board?
- A: this is a decision from Jill Burnstein (NAACL General Chair) on down.
Jon: would like to propose that we limit the number of simultaneous tracks.
Jon: how much does NAACL want to sponsor D&I?
- David: with new initiatives, it is reasonable to have expense decisions come through the board.
- Lyn: surprised treasurer was not consulted re: D&I.
Lyn: reminded everyone about CCB’s efforts to gather sponsorship to support childcare. Conclusion was not to go with that particular company again (cost a lot per child due to it being under-subscribed).
Jon: D&I is an entry in the sponsorship handbook. Sponsorship for WiNLP is a separate line - easy to get sponsorship for this.
- D&I chairs might need to be encouraged to seek more sponsorship?
- Jon: matching grants from NAACL?
Emily: would be good to have a policy to the general chair on when an expense warrants going to the exec.
Priscilla: should probably have a D&I sponsorship chair.
David: ACL to have a directory of development, NAACL should maybe have its own. Plus the new BIG (EquiCL) - too many people asking the same people for money?
- Priscilla: Which is why we have the international sponsorship committee.
David: Matt Post says - why don’t have pay what you can? Related idea: pay a registration + bonus, where bonus goes to the greater good.
Daivd: Could have an industrial rate? Based on size of the company.
Priscilla: ACL had discussion on a recruitment event with different costs based on the size of the company.
- Graham: announce how many travel grants did each company sponsor?
Priscilla: main conference sponsorship; this helps keep registration fees down. Supporting travel grants does not keep all costs down.
David: NAACL is not doing particularly well at sponsorship compared to EMNLP.
Priscilla: Options are: 3-pack, 2-pack.
- Priscilla: lots of 2-packs for ACL and EMNLP.
- One company did a 2-pack at platinum, bu silver for NAACL.
Luke: for top level, you have to be able to send a full booth. It’s not the money, it’s the people you need to send.
Priscilla: we need our NAACL sponsorship chairs to push on D&I from companies who are opting out of the higher levels of sponsorship.
Julia: how is NAACL 2019 doing?
- A: $66k surplus
Emily: do we have numbers from Vancouver yet?
- Delays due to trouble with our accounting firm
- David: for predictions, Priscilla has good estimates:
- Vancouver: $100k total split between ACL and NAACL. Very confident that it will be at least $50k split.
Jon: where does the uncertainty come from?
- A: Small-scale stuff that adds up like credit card processing fees.
- David: Cost and revenue sharing is very complicated -expect $15-$35k for Vancouver
Priscilla: 2018 is definite a surplus, likely $10k - $50k.
Jon: Where is the decision made about how many workshops will be at each place?
- Priscilla: The NAACL workshop chairs should consult with the conference organizers. She might have preferred 20, but nothing went wrong. -For both workshops and tutorials, roughly the same number of people go, and they just get spread.

Tutorial Renumeration (Priscilla)

The remuneration policy for tutorialists is based on the number of attendees.
- Old formula can result in very large payments for 100 or more attendees.
Luke: can you stop paying people to give tutorials.
Discussed and developed NAACL’s official proposal for how to deal with tutorial honorariums:
- Free conference registration for two tutorial presenters.
- Julia to bring this forward to ACL.
David: argument against free registration - people would have their company or grant pay registration.

Remote presentations (Priscilla)

Remote posters handled by notebooks + speakers
10 oral presentations, 7 posters
David: at poster sessions, there should be a sign indicating live video conferencing laptops
Google providing Chromebooks, Speakers, Instructions, software, on-site support
Impact on revenue: one author must register and pay to attend the conference in order to be included in the proceedings

NAACL 2019 (Priscilla)

1,602 attendees in total (2 more than Priscilla’s prediction)
Does not include workshop- or tutorial-onlies
90 onsite
Jon: Square versus hand-entry of credit card info for onsite?
Priscilla: current process is to hand over credit card and she runs it overnight; more secure
978 US, 53 Canada. 624 international

Treasurer’s report (Jon)

See slides.
Going to announce NAACL 2021 at the business meeting.
ACL 2020 in Seattle to be promoted at closing session.
Jon to promote 2021 at business and closing session.
Dan Jurafsky to promote ACL 2020 at closing session.

ACL 2020

Joyce to be program co-chair.
Considering early submission deadline around December.
Would also like to bring back the rebuttal period.
David: why do we need 2 months between acceptance and the conference?
- A: lining up app, handbook, website
- Joyce: 2 months is tight to get proceedings ready for handbook
- Priscilla: can no longer coordinate opening registration alongside having the conference schedule. Which makes it hard for people to register.
Joyce: Could we have some support for keeping reviewers in START from year to year? As opposed to handing off lists of emails.
Julia: Are two program chairs enough?
Priscilla: There is a budget line for a program assistant.
Graham: Senior area chairs worked quite well for ACL this year. Are there things that scale linearly with papers that can be offloaded to senior area chairs?
Julia: senior area chairs had a very hard time assigning reviewers without bidding.
- Julia: Matt Post will be able to generate a TPMs page for every author through the anthology.
- Graham: TPMs worked very well for people who were in the system.
Priscilla: People seem to like the parallel posters in daytime.
Lyn: Conference handbook is totally outdated; ACL conference officer is working on it. Need to communicate to general chair that all decisions that might impact space need to go through Priscilla, who will perhaps come back to the board.
Emily: who will own the post-conference survey this year?
- Graham volunteered.
Lyn did last year’s post-conference survey, will share with Graham.
Joyce to talk to Matt about auto-generation of TPMs.
Colin to ask Lyn to make sure survey results are stored in NAACL executive folder, linked from these minutes.
Marie to present Mexico city with support from Jon
David suggests more than one slide.

Minutes from NAACL Business Meeting

Date: June 5, 2019
Location: NAACL-HLT 2019 in Minneapolis, MN, USA

Program chair report (Christy Doran, Thamar Solorio)

Advance abstracts allowed them to react when they got more submissions than expected
- (recruit new area chairs)
- Changing abstract, title, conversion from long to short all added overhead
40 area chairs were first-timers, handled 19 papers.
Interesting breakdown of industry/grad student/professor per country
- Canada has more industry than professors
Authors were blind to area chairs
Correlation between knowing who the authors are and review score.
No author response; authors seemed unhappy about this.
Video poster highlights instead of 1-minute madness.
- One slide plus audio; lots of AV problems plagued the first day and a half.
- SRW papers integrated into sessions - better experience for the students.
- Did not repeat test of time award. When should this happen?
  - TODO: Should we make a policy outside of the conference?
START is slowing down as our numbers increase
- Revisit Open Review for *ACL?
- Looking into sharing reviews for rejected papers with the next conference.
- Reviewer overload / burnout

Remote Presentation

Assist people who cannot travel to the USA, mainly from Iran and Syria.
10 oral presentations; 7 posters
Google was supposed to supply Chrome Books?
- Colin to follow up: I did not see these in the session I chaired

Industry Track:

Focus on real-world setting and lessons learnt.
Very positive response from community on industry track.
If topic fits call for papers, please submit, even if you are not in an industry lab.
Probably no industry track at next year’s ACL.

Student Research Workshop:

NRC specifically sponsored Canadian student travel - awesome.
Pre-submission mentoring: 21 student-mentor pairs.
Piloted a non-archival track; can tick a box to avoid appearing in the proceedings.
- Idea: receive mentoring and then consider submitting to other venues.
SRW opted not to run its own mentoring service, letting the NAACL D&I handle it in place.
ACL SRW was very big this year, but NAACL has not yet seen a big increase.
Still needs more money for SRW travel awards.

Diversity and Inclusion Chairs

Started with a survey, got 58 responses, helped pick issues and form committee
- 16 volunteers.
Offered childcare grants instead of one-size-fits-all childcare.
Had a kid’s corner; email list for caregivers.
- Had difficulty finding local childcare.
$21K total in travel grants to 33 individuals (42 applicants after withdrawals).
- Late sponsorship -> expensive flights
- Sliding scale for registration fees?
- Most didn’t get the amount they wanted.
Deepmind and Google first D&I champions and ally.
Reached out to various groups to engage using Whova;
- How do we measure engagement over Whova?
Birds of a feather rooms - how can we track usage of this?
Nice to highlight editing support from Grammarly and live captions from Microsoft.
To discuss at the business meeting:
- Thoughts about the conference climate.
- Feedback on this year’s initiatives.
- What you’d like to see in the future.

NAACL Board activities (Julia)

Quick highlight of the big activities - just letting people know what’s going on.

NAACL Treasurer Report (Jon)

See slides

Upcoming Conferences

ACL 2020
- In Seattle
- Call for reviewers
NAACL 2021
- Big reaction to NAACL 2021 in Mexico City
- Excellent reveal.

Question and answer period

First time NAACLer: Other conferences require submitters to review
- Julia’s answer: Concern about experience of reviewers
- Bonnie Dorr: we have experienced reviewers, we have inexperienced ones. We need to grow the reviewer pool, which means drawing on inexperienced reviewers.
- Julia: pair senior with junior to make this work.
Suggestion: feedback from potential reviewers on how many they can handle
- Noise from the crowd: we do that.
- Julia: We do that, and a lot of people ask for reduced loads; worried about credit for amount of work done.
AAAI and IJCAI are starting to do the following:
- On a resubmission you have to provide a note explaining what happened with the previous reviews, and what has been done in response.
  - Julia: More like a journal? Good idea.

Decisions made over e-mail

Welcome and duty assignment (January 2019)

Welcome to new board members Joyce Chai, Jon May and Graham Neubig
Jon and Marie to organize ERF
Joyce and Colin to organize JSALT Summer School Scholarships
Graham commits to following through on campaign promise to look into improving reviewing

Working Group for selecting the Publicity Director (January 2019)

ACL forming a working group, which is charged with selecting an ACL Publicity Director
Luke offered to represent NAACL

Location proposals for NAACL 2021 (November 2018 to February 2019)

Marie and Joel (past treasurer) spear-headed an initiative in late 2018 to seriously consider NAACL 2021 in Latin America
Collected and discussed prospective locations in January 2019
- Initial ranking before collecting bids: Mexico City, Cancun, Ottawa

Land acknowldgement for NAACL (February 2019)

Emily raised issue of a land acknowldgement for NAACL 2019
- These are simple statements at the start of an event acknowledging the traditional Native inhabitants of the land.
Board was generally in favor, Emily was tasked with bringing the idea to Jill (NAACL 2019 General Chair)
TODO: Vote on general policy for future land acknowledgements
- Waiting for more information on how the LSA is handling the same issue before voting

2019 Outlay / Event Funding (March 2019)

Motion: NAACL will allocate the following funds to the following events in 2019.
- NAACL Emerging Regions Fund (ERF): $3,000.00
- NAACL Summer School Scholarship at the Jelinek Memorial Workshop On Speech And Language Technology (JSALT): $5,000.00
- North American Computational Linguistics Olympiad (NACLO): $5,000.00
Results: 8 yes, 1 abstain
Abstaining vote commented that the outlay budget should be larger than $13k

Discussion of Reviewing Procedures (February to March 2019)

Graham raised the issue of discussing reviewing procedures
Active, healthy discussion ensued eventually leading to:
- Looping in relevant ACL board participants (Barbara di Eugenio and Hinrich Schutze)
- Graham drafting and launching a survey - results were discussed at a pannel at ACL 2019

Trigger warning for semeval poster (April 2019)

Jon raised the following issue:
- SemEval has two tasks dealing with offensive speech and hate speech and I’m noticing that the proceedings and, ostensibly, subsequent posters, thus contain(s)/will contain lots of language that, if I was randomly walking around in a Minneapolis hotel, I wouldn’t expect to see and might be concerned about my children seeing. If I was prepared ahead of time I’d be less shocked and I expect others may feel the same way.
Asked about possibility of signs in addition to SemEval’s actions to request authors to carefully choose examples
Priscilla indicated that signage is quite do-able, issue went offline

NAACL 2021 Location (May 2019)

Motion: NAACL 2021 will be held in:
- Mexico City, June 6-11, 2021
- Cancun, June 9-14, 2021
Results: Mexico City 9, Cancun 1

Concern Regarding COI in Reviewing (July 2019)

Raised by Graham, prompted by discussion over social media
Julia/Emily assured us that the ACL Exec is aware and already looking into better COI tracking and a registry for non-expiring COIs