Objects

Since machine learning deals with unknown object attributes, a natural starting point is to talk about objects.

For purposes of this tutorial, an object is anything that has attributes.

I’ve promised that I will make abstract notions concrete and use visuals where it seems appropriate. Here is a photo that shows three exemplary objects (of desire):

Fig. 1: Image by Rawan Hussein

Among other attributes, each of these scoops of ice cream has a characteristic taste, a range of colors it is presented in, a certain level of popularity, a price it sells for, etc.

(An attribute that three three scoops of ice cream share is that they have more interesting attributes than the waffle cone.)

Attributes

An attribute is something that characterizes an object.

Let me warn you ahead of time. I love concept trees. Consequently, you will see many of them over the course of this series. Here is the first one:

Fig. 2

At a high level, we can distinguish between measured attributes and unmeasured attributes.

Many attributes are measurable. Simple examples include the weight of a scoop of ice cream and the price at which a unit is sold at a particular place and time. Out of the attributes that are measurable, only a fraction will actually be measured as part of the data collection activities during a project.

Other attributes will not be measured because they are either irrelevant, beyond our current understanding or outside of our budget. The brain, presumably, has attributes that we could measure to create the perfect new ice cream flavor. In practice, however, such an effort is constrained by our imperfect understanding of how the brain generates the subjective experience of tasting delicious ice cream and the price tag put on the required scanning equipment.

Measured attributes

Let’s further break down measured attributes into two sub-types:

Targets

Features

The target is the attribute that we care about and that we would like to predict.

Some of terms that are largely synonymous are dependent variable, unobserved variable, explained variable, output variable and outcome measurement.

[I should note that some of the concepts used in machine learning are known under different names in other communities. In many ways, machine learning is a descendant of statistics and there is a strong overlap between the two fields. Consequently, you may encounter variants that are popular with statisticians as you dive deeper into the literature.

I will mention some of the frequently used synonyms when a new term is introduced and then consistently use the variant that is most popular within the machine learning community and fits in best with the overall picture.]

Often times, the target is known for some objects and unknown for others. For example, we might want to predict the unknown future sales figures for a product based, in part, on known historic sales figures.

Features are attributes that are (potentially) related to the target.

The price of a product, for example, is related to the demand for the product. This relationship may be strong or weak depending on the circumstances[3], but price is certainly one of the first features that should come to mind.

In other communities, features are often referred to as independent variables, observed variables, explanatory variables and input variables.

Attribute values

An attribute value is the value of an attribute with regard to a particular object.

If target are the sales figures for the next quarter, an example of a target value could be 8,000 units.

Analogous to targets, we can use the term feature value to refer to the value of a specific feature with regard to a particular object. A value of the price feature in US dollars might be 99.99.

Pattern recognition

Machine learning is based on the premise that there are relationships between features and targets that repeat in a predictable manner. Let’s refer to these relationships as patterns.

If we were living in a world without patterns, there would be no use for machine learning and this tutorial would neither have been written nor read. Luckily, the universe that we find ourselves in is highly structured. The success of empirical science[4] is a testament to the idea that patterns exist and that they can be discovered.

In the not-so-distant past, Herculean efforts were made to engineer features for specific tasks. For example, there is a large literature on features that were developed for certain computer vision problems, such as the ability of robots to recognize objects in a room.[5]

Over the last decade, progress in machine learning has made it possible to automate the search for features to some extent.[6]

An Example from Computer Vision

To wrap up this article, let’s use a concrete example to gain more insight into the relationship between features and targets.

Fig. 3: Photo by Patrick Berry / CC BY-SA 2.0

Suppose you’ve just started to work on a system with the objective to automatically recognize animals in photos.

The photo above has the attribute of showing an orangutan mother and her infant, a fact that is obvious to us.

To the computer vision system, however, this is far from obvious. At the beginning of the development, it does not have any knowledge about orangutans, mothers or infants.

Before I describe what we can do to change this, I would like to ask you a question.

How do you solve this task? That is, how do you understand that the photo above shows two orangutans?

Human vision (and primate vision in general) is so efficient and occurs with so little conscious effort that it can take a while to come up with a precise and compelling answer.

I would suggest you pause and think about it for a moment. In doing so, you might anticipate some of ideas described below and eventually realize that at least some forms of machine learning are more intuitive than they may seem.