8 Comments
User's avatar
⭠ Return to thread
William's avatar

For anyone who wants to dive into the appraisal lawsuit, the complaint filed by the plaintiffs is available for free here: https://www.courtlistener.com/docket/64901934/connolly-v-lanham/ The initial appraisal was done in June 2021. The second appraisal was done in January 2022.

Below is a tangent not really related to the lawsuit or anything from the episode:

It drives me nuts that newspapers don't usually include court filings with the articles they run. They used a copy of it to write the story, why not share it? If this document wasn't on Courtlistener, someone wanting to read it would have had to spend $3 on PACER (the federal court website) for a copy. If you ever use PACER, please consider installing the RECAP browser extension available here: https://free.law/recap . It automatically adds documents you download from PACER to the Courtlistener website so others can access them for free.

Expand full comment
Thorby Baslim's avatar

Having recently read Noise by Kahneman, Sunstein, and, uh, the other guy, I wonder if noise audits have ever been done on house appraisals. Seems like a place where you would expect a large amount of noise, in addition to whatever possible bias might be present. I.e. maybe they're just BS in general? Any skeptical real estate experts out there?

Expand full comment
Dan the Peon's avatar

Quite plainly this incident is an anecdote at best, and cherry picked at worst. It's shocking that an article can be written about it given that journalism is supposed to adhere to some sense of objectivity (in the scientific sense).

When Jesse says "it's complicated", he really means (and should say!) "there are many plausible confounding factors" that need to be investigated before a conclusion can be drawn.

Expand full comment
Tyler's avatar

I wish I could take credit for it but someone on the interwebs made a very astute observation about these "racist" appraisals:. None of the aggrieved ever file a formal complaint with their state's appraisal board. If they did so, a panel of appraisal experts would evaluate both appraisals side-by-side to determine incompetence, negligence, malfeasance etc.

Instead, the aggrieved first run to politically-motivated HUD apparatchiks, or even worse, civil courts.*

*note: it doesn't matter who occupied the White House. The bureaucrats out in the field have well-oiled machine running. Political.appointees come and go. Bureaucrats live like vampires.

If every aggrieved homeowner was first required to submit a complaint to the state board, it's possible these stories might magically disappear.

Expand full comment
srynerson's avatar

I've never heard of a specific "noise audit" in connection with a real estate appraisal, but appraisals will definitely mention nearby heavy traffic areas, airports, etc. that are noise sources.

Expand full comment
Thorby Baslim's avatar

To clarify: here I meant "noise" in the sense of "random error in judgment" as opposed to physical sound. In that sense a "noise audit" means a careful analysis of the amount, patterns, and sources of inaccuracy of a certain judgment.

In this case such an audit would analyze a large number of house appraisals (ideally with multiple simultaneous appraisals of the same house by different appraisers) to see how much variance there is in the numbers.

One of the points Kahneman et al make in their book is that in many important judgments people and institutions make (e. g. judges making sentencing decisions) there is a large amount of unwarranted variation that isn't biased -- not consistently slanted for or against some group -- but still leads to unfairness or inefficiency.

So that means that when you see a discrepancy between two judgments that seems unreasonable, it could well be due largely to noise -- random variation, or variation due to extraneous factors (e. g. time of day, whether the judge was tired, etc.) rather than (or in addition to) bias based on group membership. If you have a baseline estimate of how much noise tends to appear in a certain kind of situation, that helps tell you how likely it is for a particular discrepancy to truly be due to bias. Hence the value of a noise audit.

Expand full comment
disinterested's avatar

If I was a judge in this case, I would ask why a third appraisal wasn’t done to rule this exact scenario out. Home appraisal is about as inexact a science as could be imagined.

Expand full comment
Dan the Peon's avatar

To add to this: any scientific approach must as its first step, even before looking at possible factors, check the *reproducibility* of the observations -- i.e. measure the noise.

You can have the most insightful factors and hypotheses in the world, but if your measurements are noisy, you will be able to conclude precisely nothing, unfortunately.

Expand full comment