ARCore Cloud Anchors ... Googles AR Cloud?

At Google I/O today Google announced ARCore Cloud Anchors among other updates to ARCore. The purpose is to simplify creating shared experiences:

Many things in life are better when you do them with other people. That’s true of AR too, which is why we’re introducing a capability called Cloud Anchors that will enable new types of collaborative AR experiences, like redecorating your home, playing games and painting a community mural—all together with your friends. You’ll be able to do this across Android and iOS.

This is one of the big goals of people pushing the idea of a crowd-sourced ARCloud, and probably the most important one: multi-user experiences are going to be critical to the success of AR (and VR). Cloud Anchors (as described) don't address the other goals of the ARCloud fans (long term persistence and precise geospatial positing, for example), but those may not be far behind.

The question, of course, is "what data is Google collecting, how is it being stored and what are they using it for?"^[1] I've alluded to my concerns about the privacy implications of the ARCloud, and Google doesn't come out and say if they are or are not using this data to build a crowd-sourced model of the world (the fantasy of the ARCloud proponents). It would be very nice if they came out and were explicit about what they are doing; it would be even better if they explicitly said they weren't keeping or aggregating data.

A quick reading of their Cloud Anchor documentation for iOS points to some concerning recommendations:

General

• Avoid hosting or resolving cloud anchors on flat, shiny surfaces.
  • For best results, avoid reflective surfaces or surfaces without visual features, such as a blank, smooth, white wall.
• Make sure that the lighting in the room is sufficient.
• For best results, lighting conditions should be consistent between anchor host and resolve requests.
• Pass ARFrames to your GARSession before you try to host or resolve anchors.

Before hosting an anchor:

• Try to look at the anchor from different angles.
• Move around the anchor for at least a few seconds.
• Make sure you are not too far away from the anchor.

ARKit and ARCore do not have similar needs (looking from many angles, concern about visual features, avoiding shiny surfaces, consistent lighting, etc.), which leads me to wonder if they are uploading images from the camera, or generating their own features and/or models and uploading those.

Put another way: when you run ARCore (or use Google's ARCore SDK on iOS), are you helping Google build a model of the spaces you are running AR apps in?

I've talked to multiple Google employees involved in ARCore over the past year about my concerns about the ARCloud and privacy, so I'm sure Google understands the issues here. There are simple implementations of this that are privacy preserving, but may not be as robust as matching against a full ARCloud.

When I've chatted to Google folks, I've described one such approach, which may be what they are doing:

• Save the spatial data around an anchor (generated by ARKit or ARCore), keyframe images and other feature points (of the sort used by SLAM algorithms like PTAM)
• When additional clients request the anchor, they upload their corresponding structural and feature information expressed in that clients' current local coordinates
• Align additional clients' information with the stored information for that anchor if possible
• The pose of the anchor in the additional clients local coordinates is returned.

There is no need for this spatial information to be merged or used in any other way, nor should it ever be sent to the additional clients. Such a scheme does not expose the information uploaded by any of the clients to anyone aside from Google, nor does it require that information from multiple Anchors is merged.

Of course, even assuming Google provides assurances they aren't building models by aggregating this data (which I would like clarification on, as I'm sure others would), there is no guarantee they won't keep this data and change their mind: that's just what they did with Nest a few years after assuring customers that Nest and the data it has about you and your home would remain separate from the rest of Google's data^[2].