We introduce a mathematical framework, based on the L∞ norm distance metric, to describe human interactions in a visual data mining environment. We use the framework to build a classifier that involves an algebra on hyper-rectangles. Our classifier, called VisClassifier, generates set-wise rules from simple gestures in an exploratory visual GUI. Logging these rules allows us to apply our analysis to a new sample or batch of data so that we can assess the predictive power of our visualprocessing motivated classifier. The accuracy of this classifier on widely-used benchmark datasets rivals the accuracy of competitive classifiers.