Some Thanks and Thoughts on IEEE VR 2020

It seems like the so much has happened in the just over three weeks since we announced that we were cancelling the in-person meeting and taking IEEE VR 2020 completely online.  The two weeks leading up to the conference were hectic, as my co-chair Kyle Johnsen, his students, the Mozilla Hubs team, and I figured out how to take what we'd been planning as a small companion remote experience into what ended up being a >2000 person online academic conference.

Beyond this core group, our co-chairs Ed and Kiyoshi, the entire conference committee, dozens of new volunteers, and all the presenters and participants gamely dove into this experiment  and helped to make the conference a success.  I can't begin to thank them all!

When I made that blog post on March 8, I optimistically told myself I would blog regularly, to capture the ideas we had, the choices we made, and the trade-offs that led to them. Not surprisingly, given the amount of work we did, none of that happened. Kyle and I spent far too many late nights connected over Zoom, working together (and with others) to try different ideas, technologies, and software.

I hope that over the next little while, as things calm down,  I will find the time to write up some of my thoughts on the experience.  Kyle and I are planning to write up some of this more formally, and we're working with other folks at GT and UGA to do a more formal evaluation of the experience.

But for now, I simply wanted to thank people who helped, and share some closing thoughts.  

First, Some Thanks

The breadth effort to make this conference run was captured on the slides Kyle used during the closing remarks, which I share here:

So many people, so much effort, thank you all!

And Some Thoughts

Fittingly, Kyle, Ed, Kiyoshi, and I (the conference chairs) did both the conference opening and closing the same way: starting with some slides and video in Zoom, and then moving into a Hubs room where we'd laid more materials out on the walls.  Here's a glimpse of the two rooms (I'm hoping to get them moved to a more public location soon).

Looking more closely at the far wall in our "opening remarks" space, you can see the technical progression we went through.  The diagram on the left in the image below and the one in the lower middle are from my previous blog post, showing the architecture we initially planned for the online experience.

The diagram on the right is a simplified depiction of where we ended up, focused primarily on the paper, keynote, and panel presentations.

We used Slack for our main communication channel between organizers, volunteers, and attendees.  Presentations where coordinated and delivered through Zoom;  each session had the chairs and speakers in a Zoom room along with a student in Kyle's lab who was managing the streaming station. Every presentation was done via a pre-recorded video, Zoom slide sharing, or in a Hubs room built by the presenter.  (Most of the workshops, except the VR Conferencing workshop, used Zoom for everything, and streammed directly from Zoom to our video server to Hubs and Twitch. Of the 141 presentations at the main conference in our spreadsheet, 52 or ~1/3 planned to use Zoom screen sharing, 76 or ~1/2 intended to use pre-recorded talks and then Zoom for Q&A, and 11 planned to use Hubs and either Hubs or Zoom for Q&A.  2 of the Hubs ones switched at the last minute to video or Zoom.). Regardless of the technology, the presentation was streamed directly from the streaming station out to a video server running on a serverroom.net Wowza streaming engine, and from there sent on to one of many Twitch channels (we had three for the main conference, and one for each of the workshops/tutorials) and to people watching in Hubs.  Each session had a sli.do Q&A session, moderated by a volunteer.

This diagram doesn't capture the non-presentation activity in Hubs. I will dig into more of it in future posts.

Over the course of the conference, I created over 800 different rooms for talks, posters, demos, the 3D UI contest, sponsors rooms, tutorials, and social spaces.  While most of the rooms were duplicate talk viewing rooms created to handle as many people as might want to join, more than 100 were individually created spaces for the interactive sessions.  Some of this can be seen on the front page of the online experience, a few screen shots of which I've included here.

Here's a few examples of those rooms, showing one of the poster rooms and rooms created by the COLIBRI-VR demo team and Blindness Visualizer 3D UI contest team.

Beyond the official rooms we created for the conference, attendees created their own rooms for Birds-of-a-Feather meetings (more than 20, listed on the ieeevr.org BOF page) and other social gatherings.

Reflecting on My Original Motivations...

I originally joined the chorus of folks pushing for online and remote attendence at conferences because of two issues: the carbon impact of flying to conferences, and the limited access so many have to these events.  The circumstances of this year definitely mitigated the carbon costs, since nobody flew anywhere for this conference. A number of universities (in Europe, the US, and New Zealand, at least) where planning local gatherings, but all were called off.  So the carbon impact of our conference was as low as it could be.

Looking at access, IEEE VR was also as accessible as one might imagine. Registering for the event as an attendee was free (unlikely to be the case for future online academic conferences), with the only requirement being a good enough network connection to watch a video stream or join the virtual social spaces with other people. Even though we put this together quickly, I'm particularly pleased that by the end of the conference we had registered attendees from 58 countries, compared to 32 countries represented in 2019. And the proportion of women at this years conference was over 25%, up from less than 20% last year.  It's pretty clear that moving online was a positive move from an access perspective, and I'm hoping we can find ways of building on this in future conferences!

In closing, the conference was clearly a success, and while there were a few bumps along the way, I'm very happy with the how the event unfolded.

Like many, I'm anxious about the crisis we're facing, and a cancelled in-personal conference is a minor inconvenience compared to the larger impacts the coronavirus is having on so many.  But I'm hopeful that when we emerge from this crisis, what we've learned here will have some small positive impact on how we run conferences in the future.  We can't go back to business as usual, and will still need to address other urgent, systemic problems faced by our world, including the climate crisis and the unequal access to education and research opportunities faced by so many in our world.

Update 4/10/20:  added the number of presentations that used each of the 3 approaches.