The actual disambiguation method is based on some simple linear algebra formulas.

Each tagged word is related to a sliding context where . Each tagged word has graph embedding vectors (i.e. a “sense” vector) where . What we want the disambiguation method to do is to calculate a score between each embedding vector of each identified concept of each word given its sliding context. The score is calculated by performing the dot product between the graph embedding vector and the sliding context vector . The sliding context vector is calculated by summing the graph embedding vector of each word within the window:

Finally the score is calculated using:

This is the first definition of the score we want to use to begin to disambiguate concepts from the knowledge graph associated with words in a sentence. As you can read from the formula, only the first sense for each word is being selected for each word within the sliding window that creates the context. (Other variants are be explored later in this article.) However as we will see below, we will change that formulas a bit such that the score becomes easier to understand for a human by expressing it in degrees. What is important to understand here is that we use two vectors to calculate the score: the graph embedding of the concept we want to disambiguate and the vector calculated from the windowed context that sums all the graph embedding vectors of each concept within that window.

Now let’s put this method into code. The first thing we have to do is to create a lookup table that is composed of the graph embeddings for each of the concepts that exist in the KBpedia knowledge graph as we calculated with the DeepWalk algorithm above.

( def index ( ->> ( query / get-classes knowledge-graph ) ( map-indexed ( fn [ i class ] ( if-not ( string? class ) { ( .toString ( .getIRI class ) ) ( inc i ) } { class ( inc i ) } ) ) ) ( apply merge ) ) ) ( def ^ :dynamic vertex-vectors ( .getVertexVectors ( .lookupTable deep-walk ) ) )

Getting the vector for the word is as simple as getting it from the lookup table we created above:

( defn get-vector [ uri-ending ] ( when-let [ concept ( get index ( str "http://kbpedia.org/kko/rc/" uri-ending ) ) ] ( .getRow vertex-vectors concept ) ) )

To get the sense of the word (which is SandstormAsObject ) in our example sentence above, we only have to:

( .toString ( .data ( get-vector "SandstormAsObject" ) ) )

[ 0.08586349,-0.06854561,-0.14704005 ]

To get the vector of the context of the word we have to do some more work. We have to create a function that will calculate the dot product between two vectors. Then we will have to create another function that will calculate the sum of x number of vectors.

( defn dot-product-clj [ x y ] ( ->> ( interleave x y ) ( partition 2 2 ) ( map # ( apply * % ) ) ( reduce + ) ) ) ( defn angle-clj [ a b ] ( Math / toDegrees ( Math / acos ( / ( dot-product-clj a b ) ( * ( Math / sqrt ( dot-product-clj a a ) ) ( Math / sqrt ( dot-product-clj b b ) ) ) ) ) ) ) ( defn disambiguate [ line ] ( let [ tags ( re-seq # " \[ \[ ( .*? ) \] \] \ ( \ ( ( .*? ) \ ) \ ) " line ) ] ( clojure.pprint / pprint tags ) ( println ) ( loop [ i 0 tag ( first tags ) rtags ( rest tags ) ] ( let [ word ( second tag ) concepts ( last tag ) concept ( get-tag-concept concepts ) ] ( println word " --> " concepts ) ;; Disambiguate concepts ( let [ ambiguous-concepts ( string / split ( first ( string / split concepts # " :: " ) ) # " " ) ] ( doseq [ ambiguous-concept ambiguous-concepts ] ( println "a-vector: " ( get-vector ambiguous-concept ) ) ( println "b-vector: " ( get-context i tags ) ) ( println "dot product: " ( dot-product ( get-vector ambiguous-concept ) ( get-context i tags ) ) ) ( println "angle: " ( angle ( get-vector ambiguous-concept ) ( get-context i tags ) ) ) ( println ) ) ) ( when-not ( empty? rtags ) ( recur ( inc i ) ( first rtags ) ( rest rtags ) ) ) ) ) ) )

( defn dot-product "Calculate the dot product of two vectors (NDArray)" [ v1 v2 ] ( first ( read-string ( .toString ( .data ( .mmul v1 ( .transpose v2 ) ) ) ) ) ) ) ( defn sum-vectors "Sum any number of vectors (NDArray)" [ & args ] ( let [ args ( remove nil? args ) ] ( loop [ result ( first args ) args ( rest args ) ] ( if ( empty? args ) result ( recur ( .add result ( first args ) ) ( rest args ) ) ) ) ) )

The next step is to create the function that calculates the context vector, which is a sliding window of the concept associated with , and .

( defn get-tag-concept [ concepts ] ( if ( > ( .indexOf concepts " :: " ) -1 ) ( second ( re-find # " :: ( .* ) " concepts ) ) concepts ) )

Let’s see how that works. To calculate when we have to calculate which can be done with the following code:

;; s{1,1} ( def sandstormasobject ( get-vector "SandstormAsObject" ) ) ;; s{2,1} ( def blowingair ( get-vector "BlowingAir" ) ) ;; s{3,1} ( def building ( get-vector "Building" ) ) ( def c ( sum-vectors sandstormasobject blowingair building ) ) ( println ( .toString ( .data c ) ) )

[ 0.24941699,-0.0940429,-0.11848262 ]

Finally we have to calculate the score used to disambiguate the two senses of the word blows by performing the dot product between and where which can be done using the following code:

;; s{2,2} ( def windprocess ( get-vector "WindProcess" ) ) ( println "BlowingAir score:" ( dot-product blowingair c ) ) ( println "WindProcess score:" ( dot-product windprocess c ) )

BlowingAir score: 0.011248392 WindProcess score: 0.005993685

What the scores suggest is, given the context, the right concept associated with the word blows is BlowingAir since its score is bigger. This is the right answer. We can see how simple linear algebra manipulations can be used to help us automatically disambiguate such concepts. In fact, the crux of the problem is not to perform these operations but to create a coherent and consistent knowledge graph such as KBpedia and then to create the right graph embeddings for each of its concepts. The coherent graph structure gives us this disambiguation capability for “free”.

However, these scores, as is, are hard to interpret and understand. What we want to do next is to transform these numbers into a degree between 0 and 360 . The degree between the word sense’s vector and represent how close the two vectors are to each other. A degree 0 would means that both vectors are identical in terms of relationship, and a degree of 180 would mean they are quite dissimilar. What we will see later is that we can use scores such as this to drop senses that score above some similarity degree threshold.

The angle between the two vectors can easily be calculated with the following formula:

The following code will calculate the angle between two vectors using this formula:

( defn angle [ a b ] ( Math / toDegrees ( Math / acos ( / ( dot-product a b ) ( * ( Math / sqrt ( dot-product a a ) ) ( Math / sqrt ( dot-product b b ) ) ) ) ) ) )

OK, so let’s get the angle between the example we created above:

( println "BlowingAir degree score:" ( angle blowingair c ) ) ( println "WindProcess degree score:" ( angle windprocess c ) )

BlowingAir degree score: 76.77807342393672 WindProcess degree score: 82.80469144165336