AAA conflict of interest on "EV Cold Weather study"- study not credible

Discussion in 'General' started by 101101, Feb 15, 2019.

  1. Pushmi-Pullyu

    Pushmi-Pullyu Well-Known Member

    Now wait, are you saying that the driving data from the various cars is from different years... and thus different weather conditions? I thought in your earlier comment you merely meant the study showed different model years for some of the cars.

    But yeah, if the data isn't even from the same years, and considering the record-setting cold temperatures seen in much of the U.S. recently, then I agree this study is useless. I don't think we should necessarily demand that every single study conform to the highest standards of scientific skepticism (which would involve a double-blind study), nor should we necessarily insist on statistically valid sample sizes -- which would require hundreds of units of each model of car, not just one. But if they aren't even comparing time periods with the same weather conditions, then indeed any comparison isn't worth inconveniencing so many electrons.
    ;)
     
    101101 likes this.
  2. DaleL

    DaleL Active Member

    ron in new mexico wrote: "One does not in science just go about answering questions. Study is the means to find out the proof of hypothesis."

    Hypothesis = "a supposition or proposed explanation (answer) made on the basis of limited evidence as a starting point for further investigation."

    In fact, science is all about answering questions. In the AAA study, the question which was asked was what is the range of a sample of 5 EVs when driven in winter conditions?

    My impression is that you just don't like/agree with the AAA finding. As a result, you will reject any evidence which is contrary to your belief. That is the exact opposite of science. There are also people who do not believe in climate change or evolution. Alfred Russel Wallace, 200 hundred years ago, was perplexed by "flat earth believers". Flat Earthers are still with us! You do not have to believe the AAA study, but as Bob Wilson wrote: "Ignore at your own risk."
     
    bwilson4web likes this.
  3. The actual study was conducted in controlled environments. They basically dynoed a group of EV's in a lab with different temperature conditions to simulate outside exposure.
    The problem is not with the testing but the subject matter of the test. The five are some random chance group. They started with ten but then decided to go to five as they did not want to have more than one from each manufacturer. Why then did they not just secure ten from various manufacturers with different models..... is another question unanswered.

    The five tested varied in mileage from 1K to 6k. No explanation is given if they attended to the use history of the five vehicles. They are for this test assuming as they all meet various minimum requirements, they all are equal. Which they likely are not, as batteries devolve with time and use and 1K is simply not equal to 6K.
    The most notable fact is perhaps none of the five really represent the newest technology. The tesla model is a 2017 for instance. I can't speak for all but certainly these models are seeing upgrades not so much in outside surface but quite often in internals. Teslas range to my knowledge is a limit which may be extended by the factory if so desired. They did so by patch in hurricane areas this past summer. The impact of such things is just really not available to be found in this study, as so few vehicles were used and they were all used vehicles by use pattern described.

    A survey is the means virtually all use in the industry for reliability purposes assessment. CR JD Powers and the companies themselves, though some engage this and some do not so much That is coupled
    with a vehicle complaint mechanism which often operates independent of survey for recall purpose.
     
  4. DaleL

    DaleL Active Member

    The AAA study method was:

    "Methodology

    AAA conducted primary research in partnership with the Automotive Club of Southern California’s Automotive Research Center (ARC) in Los Angeles, California to understand impacts of ambient temperature on electric vehicle driving range with and without the use of the HVAC system. The vehicles were tested using the ARC’s climate controlled test cell and state of the art chassis dynamometer and data logging equipment.

    Test vehicles were selected using a pre-determined set of criteria such as availability for sale throughout the United States with a minimum EPA estimated driving range of 100 miles. One vehicle per manufacturer was tested to prevent overrepresentation of a single brand. Additional information on methodology can be found in the full report here. (https://publicaffairsresources.aaa.biz/download/13244/)"
     
  5. You are entitled to your impression. If I held bias in this thing, I certainly would not publish which I have done here, the DOE's materials which show as fact a loss of 32-34 percent in 20F conditions for hybrid vehicles.

    Yes a hypothesis is proposed, in this it would be we find cold affects EV range. We need to examine it. A further investigation is then to find out the persistence of this and the extend of this. Is it found consistently and to what degree does this range loss extend?
    This due to so few vehicles used, no newer vehicles by model selection, and only reflective of used vehicles with widely varying use patterns, no controls used, no blinds in study endeavored at all, shows us not much at all. A BMW owner who has one of these five actually studied on this thread, is totally satisfied with it. But that it has some slight application does not mean it serves in any manner as a scientific study nor that it represents good science.
    I am totally OK with finding DOE to make a statement on EV loss of range due to cold, and expect that is if they did that found by their own scientific study. But this is not that. AAA is not qualified to make these studies nor to propose they are anything more than interesting findings like any a reporter may find.
    I have no problem with the results at all. Older EV's are probably not like brand new EV's, the technology is evolving so rapidly. If the DOE studies 17 year vehicles, my guess is the results may not apply to 19 year vehicles in many forms. Likely my guess is 17 year vehicles they find this approximate found here in this thing by AAA, as they did with earlier hybrid vehicles.
    But this by AAA is not the DOE and their qualification to produce and verify study.
     
  6. Pushmi-Pullyu

    Pushmi-Pullyu Well-Known Member

    Again, I think you're demanding too much control over conditions. 6000 miles shouldn't be a significant amount of wear-and-tear on any car which has an 8 year or 10 year warranty, as I presume all the cars in this study do.

    If the study uses dynamo testing and simulated weather conditions, then that makes it much more of a scientifically valid test than I expected. I have to agree with DaleL here: It appears that it's not that the study is scientifically invalid, it's just that you don't like the conclusions.

    You need to learn a lot more about JD Powers' methodology before you cite them as a good example of an unbiased study method. JD Powers' awards have often been accused of being a "pay to play" scheme, where only those who pay their fees are eligible to compete for the so-called "award", and those who do win have to pay even more to use the award in their commercials.

    JD Powers is, in fact, the poster child for a complete and utter lack of any scientific validity in a so-called "study".

    From Consumer Reports: "Can You Trust Those Awards You See In Auto Ads?"

     
  7. They started with ten went down to five. Yes I know their methodology. The ARC is a subsidiary, they have operational commonality with the AAA. As such in a scientific fashion for study they cannot be considered as independent as it inferred.
    They tested the vehicles in a scientific manner with good equipment. The problem is with the amount of vehicles tested it can clearly be corrupted by any technological flaw which may not be present by identified gross perameters.
    How many people buy lemons. Cars that meet basic criteria but are just not right. Quite a few actually which is why there are lemon laws in every state. But each vehicle still meets the minimum requirement for sale. They were sold legally despite not being quite right.
    This is but one exception mentioned. If only one of these five are a lemon your result are 20% off.

    Which is why you do not conduct automotive studies on reliability on such a small subject group, never.
    And if you did such a thing you would not choose at random different model years and vastly different past use criteria.
     
  8. DaleL

    DaleL Active Member

    "AAA is not qualified to make these studies..." ???????????????? Why NOT? Have you even read any of the published study? It is 66 pages long. Why would anyone trust the DOE (government) without reviewing their studies? Ars Technica reported that "In 2017, the feds said Tesla Autopilot cut crashes 40%—that was bogus. Small firm gets Tesla crash data after 2-year legal battle with NHTSA, finds flawed study." "NHTSA kept its data from the public at Tesla's behest."

    "The misinformation in NHTSA's report could have been corrected much more quickly if NHTSA had chosen to be transparent about its data and methodology. QCS filed a Freedom of Information Act request for the data and methodology underlying NHTSA's conclusions in February 2017, about a month after the report was published. If NHTSA had supplied the information promptly, the problems with NHTSA's calculations would likely have been identified quickly. Tesla would not have been able to continue citing them more than a year after they were published."

    In short, Tesla LIED for nearly two years about its Autopilot safety and the Government (NHTSA) covered it up. https://arstechnica.com/cars/2019/0...esla-autopilot-cut-crashes-40-that-was-bogus/
     
    TeslaInvestors likes this.
  9. I am using them as a example for what the industry uses not as a read on the validity of their final products. Consumer reports uses that exact method vehicle survey in their reliability reports. This is exactly their means and it is chosen as it produces a valid product.
    You are using the very source which uses the means I cite as valid to discount some other source that uses a similar means.
    You do realize then you are now in debate form...trying to hold a absurd position.

    If you are not familiar here it is. CR uses vehicle reliability reports that depend upon thousands of actual owners of vehicles that respond to their surveys. They use this as determinant of reliability for all their publications on reliability. It has some negatives to it but in general it is the best way to conduct reliability survey.
    Few dispute.
    To be clear I am advocating if this was to be the subject, cold weather loss of range that is exactly the means to use to determine extend. Simply quary owners on this specific. CR could do it for you if you are to lazy.
    This is the proven way that works. They do not test reliability by testing each vehicle in controlled environments. Some products are suitable for those means autos are not. Car tires, things related to autos are tested by controlled environment means to be clear. Autos as general category no. EV battery is a essential component of a car and as such is amenable to the reliability of the car itself, not a thing like a tire. A car starter battery is adjacent to the car like a tire and is tested individually in controlled environment. A EV battery is a car component and things like that the engine are included in the car itself reliability indicator which is always determined by survey.
    Road tests are not reliability indicators but testing functionality or application of technology to real world use.
     
    Last edited: Feb 16, 2019
  10. Pretty clear you have a agenda here. You are welcome to it. I don't know personally on the specific issue you mention as it is not my concern nor interest.
    AAA has simply no authority to conduct scientific study of automotive technology. Their primary purpose is not that. Nor are they staffed with such people who may be authority and qualification initiate such study. Nor did they subject this thing they produced to a outside group which may determine the validity of their means and methodology for publication by peer review . Their prime purpose is as insurer. Battery life of EV's has nothing to do with insurance. If they cars catch on fire due to a battery , yes.
    Perhaps that would be their purview.

    Their actual study is found by going to their site. Self directing to their custom relations person expounding on this study. Then within the statement of the public relations person look for a reference to the word "Here" in one of her sentences found. Click on that and the study in PDF form will appear.
     
    Last edited: Feb 16, 2019
  11. In America we have some pretty clearly defined means to which scientific study is determined to be valid and applicable. This is basically the publication process. The national academy of science administrates as well the national academy of engineering and the national academy of medicine, which operates in study through typically the National institute of health(NIH).
    All utilize the publication process. The end realization of the process is registration for publication in the membership cited.
    Once published it is verified as real true and scientifically. Adjacent publications within the field in each speciality serve this determination as well. Each has variance but peer review content by publication source is the means endeavored to serve as qualifier for validity.
    Anyone can publish all sorts of things and as well apply for patents for all sorts of things. This means nothing in general unless one has qualification generally through publication of study. On occasion the publishing sources themselves will provide monies and participate in the study methodology on individual items. This is commonly found in meta analysis or studies of groups of studies on the same issue to provide consistant findings of result. Other groups do exist NIH EPA NIHS and this and that to numerous to mention each which have internally known of qualification within their respective fields to bypass the publication process and self publish result of study.

    AAA Is not one of these. It may as well be fox news MSNBC or state farm, doing a study. This is how absurd is that contention.
    They may provide things of interest and use, perhaps in the fields they endeavor, but they cannot qualify their own study with authority of peer review self publish and say it is science or scientific study.
    One may be a qualified engineer, a doctor, a chemist, a this or that, and have nothing to do with nor knowledge of study requirements and publication necessity in their professional field. It is specific to the development of the science within the overall field of study and really, most are just not doing that.

    That in a bit of a big nutshell is exactly why AAA is not qualified to provide such as scientific study result of the self published sort. This represented as that in media...I can only say in this media is displaying gross ignorance. Willful or not I will not claim to know, I would guess it varies .
     
  12. TeslaInvestors

    TeslaInvestors Active Member

    No worries folks. Consumer Reports confirmed the same experience.
    https://www.consumerreports.org/hyb...-car-for-a-cold-climate-double-down-on-range/
    Buying an Electric Car for a Cold Climate? Double Down on Range.
    CR's experiences in bitter temps show that electric-vehicle batteries lose a lot of juice in the cold

    Conclusions:
    1. The whole world is wrong except what comes out of Elon's murmurs.
    2. All experimental results are false except those tweeted out by Tesla's mouthpieces and Tesla Pravduh.
     
  13. No offense TI this is even worse, which is why it is serving only as advice not a reliability indication. Two vehicles and one the older Nissan not the newer one. The other a tesla so how widely applicable are their results? They within it infer the soon to be sold Kia and Hundi, last I checked they were already out there.
    But doubt if anyone will claim it a definitive scientific study. Their general statement is fact, they do loose range. How much and is it uniform with all vehicles. Simply they need statistical result or at the very least to test new models.
     
  14. I find it odd CR coupled this with the AAA report which they do. AS the AAA report is already so structurally by method faulted.
    They used vehicles in their fleet they state. Did they just jump on this to piggy back on the AAA report publicity?
    Some of the most popular brands sold, well sure but why just the two?
    In any event in the article tesla in part responds with this...."Tesla told CR that the Model 3 started off with a less-than-normal charge because of how the Model 3’s computer evaluates the conditions outside. "
    To repeat this is a reliability issue of wide application. Autos always they use CR vehicle survey for determination. Why this exception now in this? Odd.

    To their credit I find them refereeing to this as a experiment not a study. Which I can fairly live with. This is a experiment. Using two of the five vehicles only in the other thing called as study, but is firmly not, if scientific study is what is meant which all take it to be, I find less than informative. As it would but add to a already scant data base for decision making. Again used vehicles and no controls at all.

    If they had called their thing a study, I would be inclined to change my opinion on CR and the industry on whole, thinking overt bias present. I remain on the fence on that, it was present once, now I don't know. Fox news places like that I assume bias on this. Their older viewers(in vast general terms) do not like change and EV is change.
    No offense to Nissan owners this, I owned one this decade, not a EV. But nevertheless I had to very soon trade it in. Thing would not even start and the dealer would not do much, as when at the dealership the problem did not replicate. ON a remote area in Arizona on the Navajo rez I found the starting problem and then the battery as well started to die. Luckily a brief rest and it did. Remote I would have been in trouble. Could have fought them this or that, already a slight tiny leak of engine oil....just unloaded it for another brand.

    Cold affecting batteries. I did not purchase a clarity for that express reason, they will not start at around -23F. Says so in their literature if one explores it. Canadian models are heated, the battery, the US not. Gotta be a reason.
    Cold affects batteries and then in a EV range but we all already knew that. CR recommendations on experiment I would agree with, till study shows other. wise.
     
    Last edited: Feb 16, 2019
  15. TeslaInvestors

    TeslaInvestors Active Member

    Not sure why they need statistics or need to test too many cars. These are not humans that they vary from one person to another. These are cars, made from the same components and process, and are expected to be consistent, except for the defective ones.
    AAA's study also matches people's observations, even though their conditions are not as controlled as AAA's study.
    Lots of real Tesla drivers agree. Can you reduce the loss by not using heater, dressing up lin blankets or in 3 layers like Alex Roy did for his cannonball run in Model 3? Sure. But normal folks won't go to those extremes. The study is for normal usage.

    https://teslamotorsclub.com/tmc/threads/aaa-fud-winter-driving-range.142419/page-3#post-3394219
     
  16. The normal way consumers report does reliablility issues is with statistical survey of those who have ownership. It is simply how the industry has found it the most dependable source of information. Select items such as tires starting batteries are tested in controlled manners but the cars themselves not. If you do reference the material you cited they reference it as a experiment not a study.
    For cause that, study requires all or a significant number and there must be certain conditions of control which are very hard to meet in lab work. Individual manufacturers all do lab work, experiments, it is how they design products but they are specific to vehicle and then model so it is from a defined quatifiable source of information, not a broad array with different characteristic.

    Tesla I don't know really. Tesla was used in both the experiment and what they call the study. Why that. Presumedly they already knew the AAA report had been published they mention it in the article, with tesla included in it. Why did they do tesla over. So many new vehicles are out there. Probably they just saw all the press AAA received and they jumped in to get some on their site. First thing I found with your article was a request to subscribe. A hook they had a hook. AAA provided it.
    But the reference to KIA other going to be introduced vehicles. I conversed with one person right here who already had one. How complete is their science they do not even know what models are available?

    You probably have a far more complete study of tesla than I. This AAA thing and the experiment by CR perhaps they choose Tesla for a reason, or was it random, I just do not know.
    I do know junk science when I see it. As hobby, mine was for quite some time, reviewing reports of study, many hundred I have. I can identify junk it about a five minute read. AAA thing has not wide application no controls and lacks a number to be useful. CR's thing is rightfully a experiment. Interesting good to know but not speaking to the class of cars in all eventuality.
    Neither includes new models and state of the art.
    It is a concern I have heeded battery life is affected by cold. How much at what temp....who really knows?
    How about at zero or below, can the things not be used at all? We need to know that, 20F is sort of useless.
    AS the study and experiment is sort of the same useless. It can add a slight bit to what is already known.
    How better to find out real results like that....survey..Yes I tesla leaf whatever EV owner, 10F below the things do not start. 10 our of 11 owners state that it is for us then fact. We critically need to know that sort of thing. Claimed percents for fox news to gobble up and report, not so much.
     
  17. TeslaInvestors

    TeslaInvestors Active Member

    I think you are confused a little bit here. For things on consumer opinion on cars (customer satisfaction, reliability etc.), asking lots of consumers is the way. But here, this is just the cold weather testing. I don't know how asking customers helps here.

    That survey is just one thing CR does. They also do their own tests and publish results. Do you remember last year it was CR that found the Model 3 braking issue? Tesla's QA department should have found that, but Elon fired the QA division and rolled out beta cars. So, CR did the test for them and published the result, which are correct. They did not need to ask millions of Tesla owners about their braking distance. They wouldn't know, and most would be dead by the time they will know the bad way.

    So, CR is sometimes doign more than just survey. It is doing its own testing, andhelpignboth companies and buyers. A statistical survey is not needed for these.
     
  18. Thanks for your second source, a good read.
    Apparently AAA used means which are not used by the feds in their studies. They mention(abeit these are tesla fans) what appear to be some solid criticisms that expend beyond mine which I base on study design...
    Here is one.." "Range" means how far the car can drive on a full battery. AAA did not measure "range" they measured how many kWh were used for a short commute. Two completely different things, as many people on this thread have pointed out multiple times.
    If the AAA wanted to measure how far the vehicle will drive on a full battery they needed to do a different test. They didn't but yet they report that EV range is reduced by nearly half, which is totally misleading and not supported by their tests.
    And to make matters worse, the inaccuracies from the press release were parroted by the press so people not familiar with EVs who listen to the news reports have an exaggerated idea of how much range is lost in cold weather.

    This is the EPA part...
    "I don't care about my car's range because of my daily commute, I care about it because I also use the car for long trips.
    The FUD was describing their 'use case' as a range test.
    The EPA has a couple different test cycles, in your lingo a couple of different use cases. The one EPA calls a range test is a single long drive that uses the entire battery without cold soaks during the drive. AAA should have done the same but instead devised a convoluted commuting test and sent it out to the echo chamber as EV winter range FUD."

    ON CR sure they test and find things, they do experiments. I allow and agree with that. Vehicle reliability study, which driving range is certainly a necessary part of, have to be the study result of survey. This is not a critical issue such as a brake design review. That is a study of a brake design but not a scientific study similar to the AAA report being claimed. That actually is more the purview of a new car road test. The present technology is tested to see if it works in real world application. That is a big part of what they do but it is not reliability.

    What good the information if we own a kia or hundi EV, not at all. How about any of other than the five they tested in study, not much application at all especially to newer vehicles.
    Nice to know like a popular mechanics article.We may use it for something. A read on EV range due to cold unless they get much more current than that it simply is not usefull for a new car buyer.
    I would not be to concerned with what I may be confused with. No offense, but you simply do not know me nor my ability to confuse nor be confused. I do not know you as well and will not make like claim. You seem fairly intelligent to me.

    A survey provides a much wider application of data since so many more vehicles and types are involved. Keep in mind many places have makes and models where they are we may not see so much here. A general concensus statement may be made as opposed to that derived from five dated vehicles. Places like Norway it is very significantly EV. A cold place, data from there even if it is other than our models, may have direct application as it is so cold specific and so many are in use. Survey has wide application and suffers not the limitation mentioned.
    While my criticism is again on study design, this stated in your link seems to also support a valid criticism...
    "The AAA’s test was designed to mimic the equivalent of multiple short trips in the cold.
    It chose not to test long range in the cold but reported that it tested “range.” For most people, the “range” that is most important is long range, because for shorter trips they can charge at home, work or both.
    If AAA had conducted an honest range test and reported the results for both (1) repeated commute conditions in cold weather; and (2) range on full battery in cold weather OR if AAA had explained that their test was designed to mimic multiple short commutes and did not apply to long trips, I doubt people would have had many concerns about with it.
    That the report is useful to some people who have shorter commutes and don’t have great charging options is not much comfort when it will unquestionably mislead many others whose concern about range is in the context of long trips. "

    Was it framed to produce misleading conclusion. Judgeing by even the comments on this thread seems so. Most are misreadin the data to mean something else it is not.
    I hope we are not upset about apple, it being one of virtually only two in the index that depreciated with what.... a 400 point gain? Likely it was a once off, not to worry. That other stock KO though, the other one that dropped, I have heard others say there are significant problems ahead. But I don't know. Like tesla I don't trade nor study them. Just hear rumors, mostly false.
     
    Last edited: Feb 16, 2019
  19. 101101

    101101 Active Member

    Why are you reposting that garbage- critical thinking will allow you to see through that. They had a paid for shill and it was Ars Techica (tells you everything you need to know) and their shill was debunked. You're Ford shill so how is anything you write credible?
    nica
     
  20. Thought I may mention, thanks again TI, this had not occurred to me. Your link provided this idea.
    If we wanted to produce a study which shows the absolute most loss of range in a EV, what would we study.....very cold weather stop and go with short trip length and no recharge on the in betweens.
    What do we have here before us courtesy of AAA...exactly that, coupled with a headline which even overstates that finding.
    Curious. I would not say why that, but it is curious.
    I remain with my criticism based in majority with study design and lack of general application, but would not discount this line of criticism as well, it may hold merit.

    If we wanted to discuss or show real range loss(what most are concerned with) with a EV in cold temps, what would we study...clearly to full depletion at constant use, like one would find on a long trip. Which is certainly not what the AAA study shows. Curious

    I will not get to technical, but this from the "study" states how they deviated from standard procedure the EPA uses in tests of this sort. I guess the EPA despite the length of service to include probably 30 years or so by my guess did not meet their exacting standards...
    "To conduct range testing representative of naturalistic driving environments, a custom drive sequence was constructed with a combination of EPA dynamometer drive schedules as specified in Appendix B of SAE J1634. The UDDS was performed first, immediately followed by the HWFET and a ten (10) minute soak period. After the soaking period, the UDDS and US06 Driving Schedule (or Supplemental FTP) were performed in succession. Immediately following the US06, a mid-test CSC at 65 mph was driven. The distance of the CSC was specific to each vehicle and was selected such that the end-of-test CSC was about 20 percent of the distance driven throughout the entirety of the test procedure. After the midtest CSC, the UDDS-HWFET-soak-UDDS-US06 test sequence was repeated and an end-of-test CSC at 65 mph was driven until the vehicle was unable to maintain steady-state speed.

    Their feeling(seems to be) the EPA does not provide enough real world to their stated mileage results as they typically provide two results on each car.
    Naturalistic driving environment is the term used. Never heard of that one. But honestly I have never heard of a lot of things stated these days ;) My daughters they call me a dinosaur and I would not deny truth in that. Next time I be driving and all a kilter, not paying attention, I will try that one....I am providing a naturalistic driving experience. Expect it will go over like a lead balloon, but I think it worth a try. Apparently it works in scientific study here produced.
     
    Last edited: Feb 17, 2019

Share This Page