ARCore foundation block is a session, which can be viewed as a global repository that stores information about AR system state. There are 3 main config options that can be tweaked to tailor AR experience to specific use cases:

plane detection mode

light estimation mode

update mode

The official documentation does a great job describing all the config options.

In most cases we do not need all the information collected in the session, but just the most recent data about the system state. This is where frames come into play. They capture AR state and changes within a timespan. To retrieve a frame from a session we just need to invoke a method update() on a session instance:

@Override

public void onDrawFrame(GL10 gl) {

if (session == null) {

return;

}



Frame frame = session.update();

}

From a frame we can get access to the camera, average light intensity and a list of HitResults:

private final float[] viewMatrix = new float[16];

private final float[] projectionMatrix = new float[16]; @Override

public void onDrawFrame(GL10 gl) {

...

Frame frame = session.update();

Camera camera = frame.getCamera(); //Get projection matrix

camera.getProjectionMatrix(projectionMatrix, 0, FP_NEAR, FP_FAR); //Get camera matrix

camera.getViewMatrix(viewMatrix, 0); //Compute lighting from average intensity of an image

final float lightIntensity =

frame.getLightEstimate().getPixelIntensity();



List<HitResult> hitResults = frame.hitTest(position.x,

position.y);

}

A camera instance provides ViewMatrix and ProjectMatrix, which are used to render virtual content on top of a camera image. Utilizing average light intensity we can lighten/darken our objects to make them look realistic. A list of HitResults exposes a collection of collision points. When a user clicks on the screen, ARCore creates a ray to detect a potential intersection with surfaces. A HitResult can be either a feature point or a plane:

Feature points are on the left, a plane is on the right

From a performance standpoint, planes are more stable and less frequently affected by ARCore tracking adjustments. So, it is a good rule of thumb to try to retrieve a plane from a list of HitResults. If none is detected, we should work with the closest feature point. In the below snippet if a plane is not found, we get the first element from the list, which is guaranteed to be the closest feature point because the list is sorted by depth:

@Override

public void onDrawFrame(GL10 gl) {



List<HitResult> hitResults = frame.hitTest(position.x,

position.y);



if (hitResults.size() > 0) {

HitResult hit = getClosestHit(hitResults);

}

} private HitResult getClosestHit(List<HitResult> hitResults) {

for (HitResult hitResult : hitResults) {

if (hitResult.getTrackable() instanceof Plane) {

return hitResult;

}

}

return hitResults.get(0);

}

ARCore constantly adjusts planes and feature point positions, which may lead to coordinate changes and shifting by a certain offset. This is where anchors come to rescue. An anchor is used to pin a position, where a virtual object is to be drawn, in relation to a HitResult. By relying on anchors instead of the world’s coordinates for object rendering, we can avoid major tracking issues and drifting effects.