Geometric Clustering Using the Information Bottleneck Method

Abstract: We argue that K–means and deterministic annealing algorithms for geometric clustering can be derived from the more general Information Bottleneck approach. If we cluster the identities of data points to preserve information about their location, the set of optimal solutions is massively degenerate. But if we treat the equations that define the optimal solution as an iterative algorithm, then a set of “smooth” initial conditions selects solutions with the desired geometrical properties. In addition to conceptual unification, we argue that this approach can be more efficient and robust than classic algorithms.

Susanne Still, William Bialek and Léon Bottou: Geometric Clustering Using the Information Bottleneck Method, Advances in Neural Information Processing Systems 16 (NIPS 2003), Edited by Sebastian Thrun, Lawrence Saul and Bernhard Schölkopf, MIT Press, Cambridge, MA, 2004.

nips-2003b.djvu nips-2003b.pdf nips-2003b.ps.gz 

@incollection{bottou-still-2004,
  author = {Still, Susanne and Bialek, William and Bottou, L\'{e}on},
  title = {Geometric Clustering Using the Information Bottleneck Method},
  booktitle = {Advances in Neural Information Processing Systems 16 (NIPS 2003)},
  editor = {Thrun, Sebastian and Saul, Lawrence and Bernhard {Sch\"{o}lkopf}},
  publisher = {MIT Press},
  address = {Cambridge, MA},
  year = {2004},
  url = {http://leon.bottou.org/papers/bottou-still-2004},
}