Spherical K-Means for Content Clustering

Massive needed auto-generated reading threads. I had 200 published pieces, each with a 1024-dimension Voyage-3 embedding, and a connection graph between them. The goal: cluster thematically similar pieces, then build reading paths through each cluster using the connection edges.

Standard k-means with Euclidean distance doesn't work well here. Text embeddings from models like Voyage live on a high-dimensional unit sphere. Two pieces about anxiety with slightly different magnitude vectors might be far apart in Euclidean space but nearly identical in meaning. Cosine similarity captures this, Euclidean distance doesn't.

The spherical variant

Spherical k-means replaces two things in standard k-means:

1. Distance metric: cosine distance (1 - cosine similarity) instead of Euclidean distance
2. Centroid update: after averaging cluster members, L2-normalize the centroid back onto the unit sphere

That's it. The rest of the algorithm is identical. Assign each point to its nearest centroid, recompute centroids from assignments, repeat until stable.

The normalization step matters because averaging vectors pulls the centroid toward the origin. Without re-projecting onto the unit sphere, centroids drift to shorter and shorter vectors over iterations, making distance comparisons meaningless.

function l2Normalize(v: number[]): number[] {
  let norm = 0;
  for (const x of v) norm += x * x;
  norm = Math.sqrt(norm);
  if (norm === 0) return v;
  return v.map(x => x / norm);
}

K-means++ initialization

Random centroid initialization is the classic k-means failure mode. Two initial centroids landing in the same dense region means one cluster gets split and another gets merged. K-means++ fixes this by choosing subsequent centroids with probability proportional to their squared distance from the nearest existing centroid. Points far from all current centroids are more likely to be chosen, spreading the initial seeds across the space.

The probabilistic selection uses a weighted random walk. For each candidate point, compute its minimum distance to any existing centroid, square it, and use that as the selection weight. This biases toward spread without being fully deterministic.

Random restarts

K-means finds local optima, not global ones. Running 3 restarts with different random seeds and keeping the result with the lowest total inertia (sum of distances from each point to its assigned centroid) is cheap insurance. With 200 points and k=40, each run converges in under 20 iterations, taking single-digit milliseconds in total. No reason not to restart.

Choosing k

I default to ceil(pieceCount / 5), targeting roughly 5 pieces per cluster. This is a heuristic, not theory. Too many clusters means most won't have enough connected pieces to form a viable reading path. Too few means the themes blur together.

The minimum viable thread in Massive is 3 connected pieces. After clustering, I find the largest connected component within each cluster (using the existing connection graph), then greedily build a path through it. Clusters where the connected component has fewer than 3 pieces get dropped. With k=40 and 200 pieces, roughly 6-9 clusters produce viable threads. That's plenty for a homepage.

What surprised me

The clustering quality was better than expected without any tuning. With k=40, the resulting clusters had cohesion scores (average pairwise cosine similarity within each thread) between 0.45 and 0.70. The Voyage-3 embeddings do a lot of heavy lifting. Pieces about anxiety cluster together. Pieces about attention cluster together. The algorithm just finds what's already there in the embedding space.

The connection graph within clusters is also denser than random chance would predict. Pieces that are semantically similar tend to have more connections between them, which makes the greedy path-building step more effective. The clustering and the connection graph reinforce each other.