K-means clustering

K-means is iterative arithmetic built on a circular definition: the centres decide which records belong to each group, and those records then decide where the centres move. It drops k centre points into the data, works out which records fall closest to which centre, moves each centre to the average position of its members, and repeats until nothing moves any more. What comes out is every record labelled with a group number, and a centre describing what each group is like. It is the canonical example of unsupervised learning, because at no point is it told what any group means.

In security it is a workhorse for exploratory work. Group users by how they actually behave rather than by the department they belong to, and the peer groups that emerge are frequently not the ones on the org chart. Cluster a large volume of alerts and the handful sitting furthest from any centre are the ones worth an analyst’s time. None of this requires knowing beforehand what you are looking for.

The catch is the k itself. You choose it, and that choice manufactures the answer. Ask for three groups and you get three, whether or not the data contains three of anything. Ask for twenty and you get twenty. The algorithm has no way to tell you the structure you requested is not there, and it will report a confident, tidy partition of pure noise. It also assumes groups are roughly round and similar in size, which real behavioural data frequently is not. Treat a cluster as a hypothesis worth investigating rather than a finding. It is a question to put to an analyst, never a verdict to act on.