DBSCAN


Density-based spatial clustering of applications with noise is a data clustering algorithm proposed by Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu in 1996.
It is a density-based clustering non-parametric algorithm: given a set of points in some space, it groups together points that are closely packed, and marks as outliers points that lie alone in low-density regions.
DBSCAN is one of the most commonly used and cited clustering algorithms.
In 2014, the algorithm was awarded the Test of Time Award at the leading data mining conference, ACM SIGKDD., the follow-up paper "DBSCAN Revisited, Revisited: Why and How You Should Use DBSCAN" appears in the list of the 8 most downloaded articles of the prestigious ACM Transactions on Database Systems journal.
Another follow-up, HDBSCAN*, was initially published by Ricardo J. G. Campello, David Moulavi, and Jörg Sander in 2013, then expanded upon with Arthur Zimek in 2015. It revises some of the original decisions such as the border points, and produces a hierarchical instead of a flat result.

History

In 1972, Robert F. Ling published a closely related algorithm in "The Theory and Construction of k-Clusters" in The Computer Journal with an estimated runtime complexity of O. DBSCAN has a worst-case of O, and the database-oriented range-query formulation of DBSCAN allows for index acceleration. The algorithms slightly differ in their handling of border points.

Preliminary

Consider a set of points in some space to be clustered. Let be a parameter specifying the radius of a neighborhood with respect to some point. For the purpose of DBSCAN clustering, the points are classified as core points, reachable points and outliers, as follows:
  • A point is a core point if at least points are within distance of it.
  • A point is directly reachable from if point is within distance from core point. Points are only said to be directly reachable from core points.
  • A point is reachable from if there is a path with and, where each is directly reachable from. Note that this implies that the initial point and all points on the path must be core points, with the possible exception of.
  • All points not reachable from any other point are outliers or noise points.
Now if is a core point, then it forms a cluster together with all points that are reachable from it. Each cluster contains at least one core point; non-core points can be part of a cluster, but they form its "edge", since they cannot be used to reach more points.
Reachability is not a symmetric relation: by definition, only core points can reach non-core points. The opposite is not true, so a non-core point may be reachable, but nothing can be reached from it. Therefore, a further notion of connectedness is needed to formally define the extent of the clusters found by DBSCAN. Two points and are density-connected if there is a point such that both and are reachable from. Density-connectedness is symmetric.
A cluster then satisfies two properties:
  1. All points within the cluster are mutually density-connected.
  2. If a point is density-reachable from some point of the cluster, it is part of the cluster as well.

    Algorithm

Original query-based algorithm

DBSCAN requires two parameters: ε and the minimum number of points required to form a dense region. It starts with an arbitrary starting point that has not been visited. This point's ε-neighborhood is retrieved, and if it contains sufficiently many points, a cluster is started. Otherwise, the point is labeled as noise. Note that this point might later be found in a sufficiently sized ε-environment of a different point and hence be made part of a cluster.
If a point is found to be a dense part of a cluster, its ε-neighborhood is also part of that cluster. Hence, all points that are found within the ε-neighborhood are added, as is their own ε-neighborhood when they are also dense. This process continues until the density-connected cluster is completely found. Then, a new unvisited point is retrieved and processed, leading to the discovery of a further cluster or noise.
DBSCAN can be used with any distance function. The distance function can therefore be seen as an additional parameter.
The algorithm can be expressed in pseudocode as follows:
DBSCAN
where RangeQuery can be implemented using a database index for better performance, or using a slow linear scan:
RangeQuery

Abstract algorithm

The DBSCAN algorithm can be abstracted into the following steps:
  1. Find the points in the ε neighborhood of every point, and identify the core points with more than neighbors.
  2. Find the connected components of core points on the neighbor graph, ignoring all non-core points.
  3. Assign each non-core point to a nearby cluster if the cluster is an ε neighbor, otherwise assign it to noise.
A naive implementation of this requires storing the neighborhoods in step 1, thus requiring substantial memory. The original DBSCAN algorithm does not require this by performing these steps for one point at a time.

Optimization Criterion

DBSCAN optimizes the following loss function:
For any possible clustering out of the set of all clusterings, it minimizes the number of clusters under the condition that every pair of points in a cluster is density-reachable, which corresponds to the original two properties "maximality" and "connectivity" of a cluster:
where gives the smallest such that two points and are density-connected.

Complexity

DBSCAN visits each point of the database, possibly multiple times. For practical considerations, however, the time complexity is mostly governed by the number of invocations. DBSCAN executes exactly one such query for each point, and if an indexing structure is used that executes a neighborhood query in, an overall average runtime complexity of is obtained. Without the use of an accelerating index structure, or on degenerated data, the worst case run time complexity remains. The = -sized upper triangle of the distance matrix can be materialized to avoid distance recomputations, but this needs memory, whereas a non-matrix based implementation of DBSCAN only needs memory.

Advantages

  1. DBSCAN does not require one to specify the number of clusters in the data a priori, as opposed to k-means.
  2. DBSCAN can find arbitrarily-shaped clusters. It can even find a cluster completely surrounded by a different cluster. Due to the MinPts parameter, the so-called single-link effect is reduced.
  3. DBSCAN has a notion of noise, and is robust to outliers.
  4. DBSCAN requires just two parameters and is mostly insensitive to the ordering of the points in the database.
  5. DBSCAN is designed for use with databases that can accelerate region queries, e.g. using an R* tree.
  6. The parameters and ε can be set by a domain expert, if the data is well understood.

    Disadvantages

  7. DBSCAN is not entirely deterministic: border points that are reachable from more than one cluster can be part of either cluster, depending on the order the data are processed. For most data sets and domains, this situation does not arise often and has little impact on the clustering result: both on core points and noise points, DBSCAN is deterministic. DBSCAN* is a variation that treats border points as noise, and this way achieves a fully deterministic result as well as a more consistent statistical interpretation of density-connected components.
  8. The quality of DBSCAN depends on the distance measure used in the function. The most common distance metric used is Euclidean distance. Especially for high-dimensional data, this metric can be rendered almost useless due to the so-called "Curse of dimensionality", making it difficult to find an appropriate value for ε. This effect, however, is also present in any other algorithm based on Euclidean distance.
  9. DBSCAN cannot cluster data sets well with large differences in densities, since the -ε combination cannot then be chosen appropriately for all clusters.
  10. If the data and scale are not well understood, choosing a meaningful distance threshold ε can be difficult.
See the section below on extensions for algorithmic modifications to handle these issues.

Parameter estimation

Every data mining task has the problem of parameters. Every parameter influences the algorithm in specific ways. For DBSCAN, the parameters ε and ' are needed. The parameters must be specified by the user. Ideally, the value of ε is given by the problem to solve, and ' is then the desired minimum cluster size.
  • MinPts: As a rule of thumb, a minimum ' can be derived from the number of dimensions D in the data set, as ' ≥ D + 1. The low value of ' = 1 does not make sense, as then every point is a core point by definition. With ' ≤ 2, the result will be the same as of hierarchical clustering with the single link metric, with the dendrogram cut at height ε. Therefore, ' must be chosen at least 3. However, larger values are usually better for data sets with noise and will yield more significant clusters. As a rule of thumb, ' = 2·dim can be used, but it may be necessary to choose larger values for very large data, for noisy data or for data that contains many duplicates.
  • ε: The value for ε can then be chosen by using a k-distance graph, plotting the distance to the k = -1 nearest neighbor ordered from the largest to the smallest value. Good values of ε are where this plot shows an "elbow": if ε is chosen much too small, a large part of the data will not be clustered; whereas for a too high value of ε, clusters will merge and the majority of objects will be in the same cluster. In general, small values of ε are preferable, and as a rule of thumb only a small fraction of points should be within this distance of each other. Alternatively, an OPTICS plot can be used to choose ε, but then the OPTICS algorithm itself can be used to cluster the data.
  • Distance function: The choice of distance function is tightly coupled to the choice of ε, and has a major impact on the results. In general, it will be necessary to first identify a reasonable measure of similarity for the data set, before the parameter ε can be chosen. There is no estimation for this parameter, but the distance functions needs to be chosen appropriately for the data set. For example, on geographic data, the great-circle distance is often a good choice.
OPTICS can be seen as a generalization of DBSCAN that replaces the ε parameter with a maximum value that mostly affects performance. MinPts then essentially becomes the minimum cluster size to find. While the algorithm is much easier to parameterize than DBSCAN, the results are a bit more difficult to use, as it will usually produce a hierarchical clustering instead of the simple data partitioning that DBSCAN produces.
Recently, one of the original authors of DBSCAN has revisited DBSCAN and OPTICS, and published a refined version of hierarchical DBSCAN, which no longer has the notion of border points. Instead, only the core points form the cluster.