The specific problem is with an onboard NAND-based flash storage chip, the eMMC, which is embedded in some models of Tesla’s Media Control Unit (primarily MCUv1, though MCUv2 may be affected to a lesser degree). Experts told both sites that Tesla’s cars are writing so many logs to the chip that the cars are wearing out the flash chips – an issue that is technically inevitable , but is happening a lot sooner than it should. The chip’s failure screws with functions on the Tesla’s touchscreen, which users on forums wrote could make driving the car inconvenient or even slightly dangerous (issues claimed include loss of the ability to defrost windows or charge the car).
057 Technology’s Jason Hughes told Inside EVs that when the 8 gigabyte flash card was not an issue, but over the years “Tesla’s firmware image size has gone from about 300MB to the full 1GB maximum size.” Hughes added that while the chip is designed to minimise long-term degradation via a technique called wear levelling, in which it spreads wear across “unused sections of the flash memory to extend the effective number of write cycles available,” the onboard computer also creates “excessive” logs. Along with other data caching, this results in “no free space left for additional wear levelling to compensate for the excessive log writing,” Hughes told the site.
The result is data corruption and major issues with the car. This takes years to happen (the issue is reportedly happening in cars at least four years old), but the loss of the digital dashboard is a bigger issue for Teslas than other cars. It’s also unusual for flash memory to burn out before other parts in most electronics, Tom’s Hardware noted.
“I see it in the wild,” YouTuber Rich Benoit, who repairs Teslas, told Motherboard. “I own a shop that repairs these cars and the older vehicles are starting to show their signs of age.”
Compounding the issue: the flash chip is soldered to the MCU, meaning it can’t simply be swapped out. Motherboard reported that because Tesla did not sign a 2014 memorandum of understanding between automakers who agreed to sell parts to third party repair shops, it’s difficult to find Tesla repair shops and spare parts generally have to be salvaged from wrecks.
According to Inside EVs, Hughes said the replacement MCU costs $1,800 (£1,145) in the US (or much more in certain markets). That is a cost that would have to be paid by the consumer if the Tesla in question is out of warranty. Hughes and Cotran Consulting’s Robert Cotran told the site that they can instead replace the screwed flash chip with a larger capacity, as well as use scripts to tell the computer to write the logs to RAM instead. The latter options comes at the cost of a slight performance penalty.
While technicians are currently seeing the issue pop up on MCUv1, the excessive logging issue could eventually burn out flash chips on MCUv2 or even Model 3. Tesla CEO Elon Musk replied to Hughes on Twitter that the issue “should be much better at this point” without elaborating.
Should be much better at this point
— Elon Musk (@elonmusk) October 12, 2019
“Tesla needs to just disable syslog on all vehicles unless specifically required on a development car or to diagnose an infotainment issue on a specific car,” Hughes told InsideEV. “There are absolutely zero reasons to log hundreds of MB per day to a small built-in flash chip.”
Featured image: Tony Gutierrez (AP)