Tuesday, July 8, 2014

WSJ: Confessions of a Computer Modeler: Any model, including those predicting climate doom, can be tweaked to yield a desired result

Confessions of a Computer Modeler

Any model, including those predicting climate doom, can be tweaked to yield a desired result. I should know.

By Robert J. Caprara
July 8, 2014 7:15 p.m. ET THE WALL STREET JOURNAL

The climate debate is heating up again as business leaders, politicians and academics bombard us with the results of computer models that predict costly and dramatic changes in the years ahead. I can offer some insight into the use of computer models for public-policy debates, and a recommendation for the general public.

After earning a master's degree in environmental engineering in 1982, I spent most of the next 10 years building large-scale environmental computer models. My first job was as a consultant to the Environmental Protection Agency. I was hired to build a model to assess the impact of its Construction Grants Program, a nationwide effort in the 1970s and 1980s to upgrade sewer-treatment plants.

The computer model was huge—it analyzed every river, sewer treatment plant and drinking-water intake (the places in rivers where municipalities draw their water) in the country. I'll spare you the details, but the model showed huge gains from the program as water quality improved dramatically. By the late 1980s, however, any gains from upgrading sewer treatments would be offset by the additional pollution load coming from people who moved from on-site septic tanks to public sewers, which dump the waste into rivers. Basically the model said we had hit the point of diminishing returns.

When I presented the results to the EPA official in charge, he said that I should go back and "sharpen my pencil." I did. I reviewed assumptions, tweaked coefficients and recalibrated data. But when I reran everything the numbers didn't change much. At our next meeting he told me to run the numbers again.

After three iterations I finally blurted out, "What number are you looking for?" He didn't miss a beat: He told me that he needed to show $2 billion of benefits to get the program renewed. I finally turned enough knobs to get the answer he wanted, and everyone was happy.

Was the EPA official asking me to lie? I have to give him the benefit of the doubt and assume he believed in the value of continuing the program. (Congress ended the grants in 1990.) He certainly didn't give any indications otherwise. I also assume he understood the inherent inaccuracies of these types of models. There are no exact values for the coefficients in models such as these. There are only ranges of potential values. By moving a bunch of these parameters to one side or the other you can usually get very different results, often (surprise) in line with your initial beliefs.

I realized that my work for the EPA wasn't that of a scientist, at least in the popular imagination of what a scientist does. It was more like that of a lawyer. My job, as a modeler, was to build the best case for my client's position. The opposition will build its best case for the counter argument and ultimately the truth should prevail.

If opponents don't like what I did with the coefficients, then they should challenge them. And during my decade as an environmental consultant, I was often hired to do just that to someone else's model. But there is no denying that anyone who makes a living building computer models likely does so for the cause of advocacy, not the search for truth.

Surely the scientific community wouldn't succumb to these pressures like us money-grabbing consultants. Aren't they laboring for knowledge instead of profit? If you believe that, boy do I have a computer model to sell you.

The academic community competes for grants, tenure and recognition; consultants compete for clients. And you should understand that the lines between academia and consultancy are very blurry as many professors moonlight as consultants, authors, talking heads, etc.

Let's be clear: I am not saying this is a bad thing. The legal system is adversarial and for the most part functions well. The same is true for science. So here is my advice: Those who are convinced that humans are drastically changing the climate for the worse and those who aren't should accept and welcome a vibrant, robust back-and-forth. Let each side make its best case and trust that the truth will emerge.

Those who do believe that humans are driving climate change retort that the science is "settled" and those who don't agree are "deniers" and "flat-earthers." Even the president mocks anyone who disagrees. But I have been doing this for a long time, and the one thing I have learned is how hard it is to convince people with a computer model. The vast majority of your audience will never, ever understand the math behind it. This does not mean people are dumb. They usually have great BS detectors, and when they see one side of a debate trying to shut down the other side, they will most likely assume it has something to hide, has the weaker argument, or both.

Eventually I got out of the environmental consulting business. In the 1990s I went into a completely different industry, one that was also data intensive and I thought couldn't be nearly as controversial: health care. But that's another story.

Mr. Caprara is chief methodologist for PSKW LLC, which provides marketing programs for pharmaceutical firms.

22 comments:

  1. I built an economic and financial model for shopping centers in London England from 1964 to 1969 an IBM360 computer. The program used approximately 1,000 FORTRAN statements on punched cards and was structured with a main root section that was just a list of subroutines as overlays.

    After more than 5,000 hours work building and testing the model I concluded that it was worthless for prediction and abandoned the project.

    I did observe that a PhD candidate could tweak such a model to make it look as if it had predictive skill that it did not possess.

    ReplyDelete
  2. I should hope that a fellow environmental engineer would know that the correct term is sewage treatment plant. A sewer treatment plant would be a facility for doing something to pipes.

    ReplyDelete
  3. Modelling something as complex as climate with feedback loops where we don't have a full understanding means that you have to take short cuts. I doubt we actually have the ability to model to a point where we can make reliable predictions for the future. While I do suspect that we are warming the planet, and we are abusing our planet, proving it is a completely different matter. I studied mathematical modelling in the 80's and the whole subject matter revolved around our limitations. Somehow that has been swept under the carpet.

    ReplyDelete
    Replies
    1. "Eventually I got out of the environmental consulting business."

      There ya go.

      Delete
  4. I spent over four years trying to develop a computer model that would accurately predict detailed results of a nuclear power plant. Our team had reams of actual data from the plant we were modeling. And, we had the actual plant to play with and confirm if the changes we subjected to the plant caused the EXACT same changes to the computer model. Starting with an existing "model" that was used for a "training simulator," we spent the first two years trying to get output that was accurate to the first decimal point. This required reviewing the output data then going over the algorithms, subroutines, assumptions and "tweaking" gains, delays, integrals, etc. correcting the problem and then submitting the revised changes to run again over night or over the weekend. And then do the same thing over again. Most of the time all this did was create a problem in one of the other output parameters. And again, we knew it was a problem because we had the real, actual, plant and KNEW what we were supposed to get. Once that phase was completed, we would try more severe perturbations of the model that were allowable to be performed by the plant. This took several more years. There were some perturbations (accidents) that were never "proven." In the end we did not get a comprehensive, all inclusive model. It was more than sufficient for a very accurate "Training Simulator" but we had to rely upon other limited purpose models for some of the major accidents. We could start at any point of time/power and the results (to the normal operator) were indistinguishable from the actual plant. Years after completing this effort I was informed that during a training exercise that the "simulator model" predicted unexpected results for a multiple causality event (loss of several systems).

    We knew every parameter involved, had actual plant data accurate to 3 decimal points, knew the science for every action/result involved, had several years of data for the plant at all stages of power output, power ramps and accidents that had happened at that plant and similar plants and still the model output was not as accurate as the garbage that the AGW group is providing as output data, even if you include their "error bands."

    The AGW models are worthless. PERIOD

    ReplyDelete
  5. The shear number of different climate models is evidence that the IPCC does not really know what they are doing. A lot of guess work has been involved. Apparently they started with weather prediction software and modified it so they could make climate runs in finite time. They hard coded in that increases in atmospheric CO2 causes warming so their climate simulations beg the question.
    They fail to simulate what really happens in the atmosphere.

    I can only assume that what is coded in is what is stated in descriptions of the greenhouse effect theory. According to a plausible statement of AGW theory, adding more CO2 to the atmosphere increases the atmosphere's radiant, thermal insulation properties. I am stating the theory in the most plausible way possible. The increase in insulation causes warming in the lower atmosphere but cooling in the upper atmosphere where earth radiates to space in LWIR absorption bands. Remember a good absorber is also a good radiator so it is the greenhouse gases that actually radiate to space in LWIR absorption bands. The warming of the earth's surface and lower atmosphere cause more H2O to enter the atmosphere which according to AGW theory causes a further increase in the radiant thermal insulation properties of the atmosphere because H2O is also a greenhouse gas. The increase in radiant thermal insulation causes more warming which causes even more H2O to enter the atmosphere and so forth. AGW theory stops with the idea that H2O provides a positive feedback to the addition of other greenhouse gases and it is this positive feedback that is needed to make the warming effect significant. Apparently this warming effect to include the positive feedback is hard coded into the climate models. The problem is that this is not all what happens in the Earth's atmosphere.

    Apparently AGW theory ignores the fact that besides being the primary greenhouse gas, H2O is a primary coolant in the Earth's atmosphere, moving heat energy from the Earth;s surface to where clouds form via the heat of vaporization. More heat energy is moved by this mechanism then by both convection and LWIR absorption band radiation combined. More H2O means more heat gets moved which provides a negative feedback. Apparently the models ignore this very important mechanism.

    According to greenhouse effect energy balance theory, in LWIR absorption bands, from space, the Earth looks like a 0 degree F black body radiating at an equivalent altitude of 17K feet. But their is no black body at that altitude radiating to space. Because of the low emissivity of the atmosphere what we really have are gray bodies radiating at lower altitudes and at higher temperatures. We are talking about typical altitudes of clouds. More H2O means more clouds. Clouds not only reflect incoming solar radiation but they radiate more efficiently to space then the clear atmosphere they replace. Clouds represent another negative feedback that the simulations ignore.

    As the lower atmosphere warms, the upper atmosphere cools. That is how insulation works. By upper atmosphere, we are talking about the actual equivalent altitude where Earth radiates to space in LWIR absorption bands. The cooling causes less H2O to appear which counteracts the effects of adding CO2. This upper atmospheric effect is another negative feedback that the AGW modelers have also ignored.

    Apparently the simulation models include a lot of detail that is important to short term weather prediction but not to climate and they leave out major effects that are important to climate prediction. The models leave so much out so that their predictions are worthless and that has been their track record.

    ReplyDelete
    Replies
    1. Well said. Net positive feedback from water vapor exists only in computer code.

      Delete
  6. Was the EPA official asking me to lie ?

    YES, he was asking you to lie. And you admitted that you did.

    It's not a gray area or a "judgement call". You were asked to lie and you did.
    It's a real shame that integrity and actual common sense value and behavior judgements are so diluted in todays education and upbringing that lying and dishonesty have been RENAMED as something less and acceptable to most.
    Seriously.... a shame.

    ReplyDelete
    Replies
    1. You're right- it is a lie, whitewashed by a computer

      Delete
  7. I concur with the above commenters. I have had the pleasure of developing models for various electro-mechanical devices over a 30 year career. It basically becomes an exercise in multi-variable minimization of a cost function for a parametric representation. If the response is linear, and you can activate the device in a closed loop fashion with a wideband source of persistent excitation, running the test many times longer than the longest response time of the device, and dynamically adjusting the parameters with an appropriate multivariable descent method in a closed loop fashion, then you can generally converge on a solution which represents general performance reasonably well.

    None of those conditions hold with the climate. When any of those conditions do not hold, the odds of successfully modeling the device grow long. Often, the best you can do is get general response information which allows you to design a stable feedback loop around the device which essentially replaces the unknown dynamics with the known dynamics of your feedback loop. Unfortunately, that generally means you cannot take advantage of the full range of capability of the device, as you have to build in enough stability margin to ensure that the loop remains stable.

    Success very much depends on the fidelity of the model you need. Even the simplest devices, when you get near the limits of its capability, become wildly variable, and it becomes a game of whack-a-mole trying to push down the "tall pole in the tent", which often just makes another pole pop up to take its place.

    - Bart

    ReplyDelete
    Replies
    1. These are exactly the same as my problems. We essentially quit modelling when we could no longer justify the cost with the diminishing returns.

      Delete
  8. I really appreciate this article, more than I can tell you.

    An observation I will share is, this climate change discussions that really are fairly recent, and driven by the Al Gore dogma, began in another form.

    It began with fishery "science" pushed by the Pew Charitable Trust, and the pseudo scientists and ecologists that have greased the skids for the Pew Agenda.

    Son of a gun, wouldn't you know it?

    The ones that have become very wealthy shilling for Pew, and all the other take Eco off groups like Oceana, EDF, NRDC, and so on, have become deeply entrenched in our nation's and of course, "global" arena of fishery mismanagement.

    Computer modeling is a huge decision maker in this field, and is highly suspect in its always dooms day results.

    I've posted this article at fisherynation.com, and will certainly be bringing this to fishermen around the world. Thank you again, and please explore this website to understand the parallel.

    The Big Green Money Machine – how anti-fishing activists are taking over NOAA
    For the first time in at least a century, U.S. fishermen won't take too much of any species from the sea, one of the nation's top fishery scientists says.” This is from an article written by Jay Lindsay for the Associated Press (link) and the top fishery scientist is Steve Murawski, who retired early in 2011 as Director of Scientific Programs and Chief Science Advisor at NOAA Fisheries. So why are so-called “marine conservationists,” ENGOs, the handful of billion dollar foundations that support them and the upper echelons at the National Oceanic and Atmospheric Administration in the U.S. Department of Commerce still claiming that radical surgery is needed to “save” our fisheries? The answer to that question is beyond me, and beyond anyone who’s likely to be looking at this website. http://www.fishtruth.net/

    ReplyDelete
  9. Go to the well:

    "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk."
    -John von Neumann

    ReplyDelete
  10. I have a story very similar to the author's. In the mid-1980s while working as a research engineer at one of the Department of Energy's national labs, I was asked to write a computer model to project the potential health effects from exposure to radon gas present in homes.

    Radon detectors were placed in hundreds of homes and exposure data taken and collated based on foundation types: full-basement, daylight basement, slab foundation or crawlspace to be run through my model.

    Here is where my experience parallels Mr. Caprara's. The biologists in charge of the project simply postulated that radon gas affects the body in exactly the same way as cigarette smoke. The biologists provided me with equations curve fit from an old tobacco smoke health effects study, and had me crunch the radon exposure measurements through the tobacco exposure equations as if it were cigarette smoke rather than radon present in the homes. Their objective was to determine the postulated public risk of getting lung cancer from exposure to radon gas.

    As with the author, my results came in substantially lower than the lead researchers wanted, and I too was told to "sharpen my pencil." I was asked to add a calibration factor to the model to 'tweak' the results closer to their expectations. When the results were still not dire enough and I was told to tweak the model again, I simply made the calibration factor an input variable and told the researchers that now they could make the model give them any result they wanted. I walked off the project then and told them not to include my name on any reports or papers.

    The radon researchers went on to publish their study concluding significant public health risks from radon gas, which they were able to parlay into substantial follow-on research which eventually led to public policy and even building code changes based on what I know to be a completely bogus analysis.

    My experience made me very skeptical about the objectivity of researchers, particularly when future funding and follow on research is at stake. I learned as Mr. Caprara learned, that computer models are no more than just a sales tool for policy advocates.

    ReplyDelete
    Replies
    1. That's astonishing

      Follow the money, even in "science"

      Delete
  11. When I was in engineering grad school and first introduced to "finite element modeling", immediately around the room there was talk of its nick-name: "pink elephants". We were taught right from the start that it was a GI/GO tool.

    ReplyDelete
  12. Your ice data is incorrect there has been a processing error for antarctic data refer to sunshine hours

    ReplyDelete
  13. In 1992, as a graduate student with 20 years experience as an independent contractor, I did an analysis of the computer models then predicting Hansen's Global Warming as potentially mitigated by landscape-scale reforestation (my professional field). I presented my findings at an international conference on the topic, and had my funding eliminated as a result. That is how I learned about "consensus." I think at least four of my five Conclusions remain valid to the present time, more than 20 years later: http://www.nwmapsco.com/ZybachB/Reports/1993_EPA_Global_Warming/index.html

    ReplyDelete
    Replies
    1. That's sad to hear. No doubt many others who dared question the climate models have been sacked for dissent.

      Delete
  14. This kind of thing goes on in most sciences where there agencies that want to make "informed" decisions. An honest appraisal is provided, the report battles its way through reviewers whose sol interest seems to be insuring the official agency spelling is used throughout or questioning comma use. Finally after adding elements that are completely unnecessary to begin with simply to meet one agency's biases, the report goes on to a second who demands that all the agency specific and unnecessary changes already made be reversed and their own pet preferences be inserted instead. Quite often the level of cynicism is astonishing. The agency "manager" is not interested in an "informed" process. Instead they want the squeaky wheels greased so they'll shut up.

    ReplyDelete
    Replies
    1. Yup, pure bureaucracy and politics, disguised as science.

      Delete