BEST team member Zeke Hausfather made an interesting remark in a discussion at blogger Anders's place. A commenter had mentioned BEST's work examining the Urban Heat Island (UHI) effect, pointing out BEST hadn't found evidence artificial warming from urban development significantly impacted its results. Zeke commented:
To be fair, the separate study that Berkeley did was on homogenized data, so a lack of detectable UHI mainly just indicates that it was effectively removed.
I find this remarkable. When I discussed BEST's uncertainty calculations, I pointed out they don't redo the homogenization calculations. The result is they don't even attempt to calculate the uncertainty in their homogenization process, meaning they know there is more uncertainty in their results than they claim.
The same problem arises with their approach to looking for a UHI effect. To look for a UHI effect, BEST compared rural to non-rural stations. Only, it did so after it had already homogenized its data. That means BEST modified its rural stations by using data from non-rural stations (and vice versa) then said it couldn't find a difference between the two.
Of course they couldn't find a difference between two data sets after using those data sets to make one another more similar. I get Zeke says their homogenization "effectively removed" the UHI signal, but there's no way to know that is true. An alternative explanation is homogenization just smeared the UHI signal out a lot. We've already seen BEST show its homogenization smears signals out over a great distance:
It is not difficult to believe this smears UHI effects from non-rural stations to rural ones. Doing so wouldn't remove the UHI effect. All it would do is make it so rural and non-rural stations share the same UHI signal.
BEST homogenized rural and non-rural stations then found little difference between the two. Rather than saying, "Oh, that's what homogenization does," it said, "Clearly, UHI isn't a problem for our results."