Real-estate, AVM

In this article, we are going to discuss and analyse the European standards for Statistical Valuation Methods for Residential Properties, published by the European AVM Alliance in August 2019. Our goal is to present a summary of this guide as we understand it from a data science perspective.
In an era of Alexa, Siri, blockchain and Artificial Intelligence (AI), property valuations require manual examination of documents and some times onsite inspection by experts to conclude on the price of an asset. This is an error-prone process as usually no two experts agree on the same price.
Automated Valuation Models (AVM) aim in automating this process removing bias and subjectivity from these decisions as much as possible. This is not a new concept but as AI has seen exponential growth due to its unprecedented accuracy and performance in the last few years, AVMs gain more popularity as a consequence. Your typical tutorial on machine learning will probably provide you with a real-estate related dataset to play with. Even so, only a small fraction of real-estate businesses actually take advantage of these systems and even worse some do not even use databases to store property-related data.
The European Standards is a guide on the development of property valuation models and specifically what to consider when building such systems. Importantly however it aims at increasing the understanding, transparency and clarity of the existing Statistical Valuation Methods and the selection of the most appropriate method and evaluation approach.
As mentioned, the aim of valuation models is to estimate the value of a given property on the basis of the values of other properties, similar to how an expert would do. The guide examines four main different approaches, namely, House Price index, single parameter valuation, hedonic models and comparable-based models.
Prior to delving into the details of the modelling approaches the guide defines a few main concepts such that to make sure all readers have a common understanding of the terms used throughout. For example, a key question is what do we mean by Market Value? For a lending institution, it is the estimated amount for which the property should exchange on the date of valuation between a willing buyer and a willing seller in an arm’s-length transaction after proper marketing wherein the parties had each acted knowledgeably, prudently and without being under compulsion. For accounting is the price which a buyer would be prepared to pay for it, having due regard to its condition and location and on the assumption that it could continue to be used.
A statistical valuation method may have different uses and thus may be different in their nature. Specifically, it may require different degrees of detail, accuracy and information on the result. For instance, the accuracy of the valuation of a property on a popular UK property search website is not as crucial as the valuation of the property for a mortgage application. For some intended uses the required amount of detail may or may not be available to feed a Statistical Valuation Method.
Another concept is Mass appraisal. This is the practice of portfolio valuations, i.e., valuing multiple properties as of a given date. With all things being equal, a lesser degree of granularity sometimes associated with data in property portfolios results in lower accuracy than a greater degree of granularity. However, the amount of time required to value these properties outweighs the risk, on some occasions. Thus it makes them the perfect candidate for automation.
The guide states that models must be calibrated before use. In data science, we typically use cross-validation and hyperparameter tuning to adjust any free model parameters to better fit our data; balancing this way the risk of overfitting (i.e., the risk of the model learning only the input – extremely well – and not being able to generalise on unseen data). In the guide, this is referred to as a requirement of mathematical skill and expertise. Additionally, equally important is to have relevant and sufficient market data.
As an Avm developer, you must ensure data quality. As we say, bad data in, bad output. Also, data and perhaps the model should be updated regularly in order to reflect new transactions and capture new trends or important characteristics. The AVM has to be objective and conform to any national laws applicable and the developer must keep an open door policy for auditing the model by 3rd parties.
Another aspect covered in the guide is model performance and model evaluation. It suggests that the model must be tested and validated. Since no model is perfect its error must be quantified and understood in order to grasp its limitations. The testing should replicate real-world situations such that the results to be representative of such scenarios. The de facto data science approach to validating models is cross-validation. Cross-validation is the technique of training a machine learning model on a subset of the data and test on another subset never seen by your model before. However, you can repeat the process by taking different subsets of your data for training and testing, while never training a model on data that it will be tested later on. In other words, the aim is to thoroughly test the model in a series of out-of-sample datasets. This is a so-called test set.
The guide also stresses a point about what the model aims to predict. In Data Science, we refer to that as a target variable. This is NOT the asking price but rather the actual value of the property which is reflected in the sales price. Note that this is not always true as someone might have to sell if they need to liquidate urgently but it is the best way to assess its value.
No matter what we aim to predict we always need a robust way to measure the success of the model. The metrics for success vary, but Coverage is an important concept in the guide that captures the portion of predictions that are within an acceptable range. In other words, this is the ratio of cases producing a valid result divided by the total number of cases. Also, the guide refers to bias and dispersion which is the classical Bias versus Variance trade-off in data science and machine learning. This is about whether the model always overestimates or underestimates the price, i.e., it is biased, while at the same time is consistent with its prediction in terms of how far each prediction is from the actual price. In case the predictions are always far off by a different margin, this means the model has high variance.
Finally, the guide talks about three main techniques for a valuation model, none of which utilises Machine Learning per se but rather implies traditional statistical models. The four approaches are the House Price Index (HPI), Single Parameter Valuation, Hedonic model and Comparable-based Automated Valuation Models.
In reality, HPIs are just indices that capture an aggregated statistic about properties in some areas and it is not the right tool to value a specific property. Sometimes HPIs are just expert opinions while other times are aggregations of properties in a time period. Another HPI method is the Basket of Goods Approach. In a Basket of Goods Approach, the HPI is calculated from individual values of the same properties over time. The quality of each valuation still depends on the quality of valuation of the properties in the basket and it still depends on what properties are in the basket. Repeat Sales Indices is also an HPI that reflects the recorded Sale Prices of the same properties at two or more points in time. However, what happens if there are no many sales in a specific region or the interval between sales is long? In any case, HPI does not give a prediction for a property but rather potential changes from a reference point.
Single parameter valuations are models that aim to estimate the value of a property based on a single parameter, along with its geography. A generalisation of this is the hedonic models.
Hedonic models are multivariate regression models that have property characteristics as input and the parameters are learned from the data. This is towards a data science approach but it excludes models that do not have the form of a linear or non-linear equation.
Comparable-based AVMs are tools that select the most similar properties in relation to some reference property. This seems more like a K-nearest neighbour approach, where each property is a data point in an N-dimensional space and the aim is to identify ‘K’ other properties that match the most to the referenced one. However, defining what is a comparable can be challenging. Is it a combination of location and size or a linear combination of all the property characteristics? So what property characteristics do we look for when looking for comparables and what happens if no exact comparable exists?

In data science, we employ different algorithms and approaches to utilise the historic data we have and we do not necessarily need to choose one of these four approaches.
In strategy when you are given four options you choose the fifth one.
That is not to say that existing approaches are not useful. Indeed, they are very useful and perhaps complement each other. In addition, however, there are machine learning techniques like Random Forests and Gradient Boosting that is shown to perform well in most regression and classification problems. There are stochastic non-parametric processes that capture non-linear relationships between input and target variables and there are deep learning approaches used in the latest Artificial Intelligence applications. All of these can capture relationships that no single linear or non-linear regression model can capture, they can provide predictions for individual properties and address the issue of no direct comparable by generalising on the whole dataset the model was taught.
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a portion of your membership fee if you use the following link, at no extra cost to you.
If you are new to data science you might be also interested in the following articles: