tag:blogger.com,1999:blog-4172950626830217643.post6385331432251675353..comments2024-03-28T03:20:54.405-04:00Comments on Better Embedded System SW: Proper Watchdog Timer UsePhil Koopmanhttp://www.blogger.com/profile/11849599272360094243noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-4172950626830217643.post-11250807848339821612021-10-15T21:40:30.292-04:002021-10-15T21:40:30.292-04:00You need to do this to prevent a race condition du...You need to do this to prevent a race condition during the operation (i.e., to make it an atomic operation), assuming a single core processor.Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-52293090029772509782021-10-15T20:48:53.182-04:002021-10-15T20:48:53.182-04:00Hello, I bought your book is very interesting!!.
...Hello, I bought your book is very interesting!!.<br /><br />I have a question: <br />why in the above code inside the "Void Alive function (uint16 x);" interrupts are disabled and then enabled. It may be a silly question but I am new to the world of microcontrollers.<br /><br />ThanksLuis Peñahttps://www.blogger.com/profile/08296337096384402935noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-4879992553766085022018-09-06T09:18:59.983-04:002018-09-06T09:18:59.983-04:00You might want extra protection to ensure that onl...You might want extra protection to ensure that only authorized tasks check in with the watch dog, each task is only setting one bit, that the bit being set actually corresponds to the task making the call, etc.<br />This is a sketch of the idea. One reason you should use a commercial operating system instead of a home-made one whenever possible is because these are the types of things that the pros (hopefully) think of and handle.<br />Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-68837737745783172132018-09-06T04:37:05.568-04:002018-09-06T04:37:05.568-04:00The idea looks really great. But shouldn't wat...The idea looks really great. But shouldn't watch_flag be synchronized ? What if I have task that is not allowed to lock mutex ?Anonymoushttps://www.blogger.com/profile/01142938526741571454noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-9383869200085419302017-06-08T09:07:34.455-04:002017-06-08T09:07:34.455-04:00Hi Frankenstein,
I was just reading your posting, ...Hi Frankenstein,<br />I was just reading your posting, a bit late, but I have AUTOSAR experience. The one thing that baffles me is that persons post their problem, then usually receive an answer and then never post again if the problem was solved and even more useful to the community how it was solved.<br /><br />In principle I think you need to either have the algo moved to a different task or you change the task timing. Try 10ms. Ask your AUTOSAR integrator to change the configuration and don't forget to test things and measure your actual algo/calculation.<br /><br />RegardsBlogger Wonder 1https://www.blogger.com/profile/06789250655383423971noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-203395202347345282016-08-26T07:16:24.186-04:002016-08-26T07:16:24.186-04:00I sometimes use an regular interrupt as the monito...I sometimes use an regular interrupt as the monitor task whereby it only kicks the dog if all of the other tasks have 'checked in'. But yes, agreed, never just simply kick the dog unconditionally in the isr. I cannot agree with the position that watchdogs are un-necessary in well designed code. In complex systems there is no way to be 100% sure that some failure cannot happen. In any case, there is always the possibility of memory corruption due to harsh environment or whatever. I would never allow my team to ship a product without a watchdog - our systems are not safety critical, but if they simply locked up in the field without recovery I'd be looking for a new job. I do agree however that it is important to detect, log, and investigate watchdogs. In our systems where there is an RTOS we can use the Task Control Block to ascertain where the execution locked up and this provides a useful bread crumb to trace back to the origin of the problem.Tomaltachhttps://www.blogger.com/profile/06472288290882778889noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-44562011989620060452016-06-29T01:16:25.748-04:002016-06-29T01:16:25.748-04:00Just to add my two cents...
I've been working...Just to add my two cents...<br /><br />I've been working in the design of hardware and software for safety-related products (up to SIL3) for several years and, although I somewhat agree that watchdogs may lead some people to be lazy/sloppy at coding, I fully agree with Phil that, at least for safety-related applications, not having a watchdog is not an option, no matter how thorough and rigorous that has been the design/development process.<br /><br />Moreover, many of the comments above expressed concern about the consequences of an erratic reset of the CPU due to its failure to kick the watchdog in a timely manner. However, resetting the CPU is not the only option: a different, although quite common approach in safety-related applications (in particular when the failure of the CPU means that you can not longer rely on it), is to have an external, hardware-based watchdog circuitry that, if not refreshed appropriately, it causes a safe shutdown of the system that switches all safety outputs to their safe state.<br /><br />The safety shutdown mechanism is carried out by the hardware circuitry of the watchdog subsystem and it usually includes a latching mechanism to prevent the automatic (unsafe) restart of the application.<br /><br />Additionally, the circuitry of the external watchdog is kept as simple as possible so that its failure modes can be analyzed and taken into account and it also provides some control and feedback signals so that the CPU can check that the external hardware is operating properly. If that's not the case, now it's the CPU which initiates the safe shutdown.<br /><br />The principles of a similar approach are already described in http://betterembsw.blogspot.ca/2014/04/monitor-actuator-pair-design-pattern.html<br /><br />Obviously, this approach is not suitable for all safety related applications (e.g. you don't want something like this in the engine control of an aircraft) but still has its place in applications where the consequences of a failure can not tolerated (e.g. injury or death) or when they are costlier than the economic impact of the shutdown.<br /><br />Thanks and keep up with this excellent blog,<br />Picante.//<br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-21371893633961290822016-04-24T09:56:28.176-04:002016-04-24T09:56:28.176-04:00Do not ever disable the watchdog once your program...Do not ever disable the watchdog once your program initialization is completed.<br /><br />It might be justifiable to lengthen the watchdog period a bit if it is still fast enough to be effective (see my post on how long to set the watchdog from Nov 9, 2015). In general if you have a really long function either you need to break it up into pieces and run it a piece at a time or protect it with a much longer period software watchdog and take the risk that SW watchdog is possibly not as high a level of protection as HW watchdog.<br />Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-87073555782327561672016-04-23T22:23:14.090-04:002016-04-23T22:23:14.090-04:00HI thanks for this great article , i have made thi...HI thanks for this great article , i have made this week the first contact with the WDT after debugging a bug in my code but i'm having the following issue if you could help plz<br /><br />i'm working on a project for automotive system where we use the MPC5748 MCU ... the application use an RTOS based on AUTOSAR OS , and this MPC target support two type of watchdogs : software and hardware (they have used soft WDT).<br /><br />My mission is to fit an algorithm within this application,the developpement of the algorithm has been done ... the problem is that in the task where the algorithm is running is a 1ms task and the algorithm needs much more time than the time dedicated to this function.<br /><br />I'm a newbie to the embedded world.By the way, in the algorithm main function the program will reset itself and this seems to be a timeOut generated by the expiration of the watchdog.<br /><br />My questions are:<br /><br />Can i disable the wachdog timer for this specified function (wich must not be disabled but just for testing purpose) ? It is possible to use more timeOut for the watchdog on that specified function. ?<br />Must i develop another task with a big delay in orther to run the algorithm , but the problem is that the algorithm need to be synchronised with the 1ms task since we are receiving can commands ...<br />What are other options to try ?<br />NB: This is a general problem on the watchdog timer and any useful informations will be much helpfull for me.Sorry because i can't share the code !!!<br /><br />thanksAnonymoushttps://www.blogger.com/profile/07962293664282088927noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-19879297968089941042016-04-09T18:54:46.722-04:002016-04-09T18:54:46.722-04:00Thank you for your post. I am an electronics engin...Thank you for your post. I am an electronics engineer in a small company, in a small company I need to do everything including writing firmware. I don't have a degree and mostly self-taught. Most of our systems are not critical, but your post makes me a better embedded Software designer. <br /><br />Thank you!<br /><br />James WuAnonymoushttps://www.blogger.com/profile/06656630580678519233noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-90747864213264131742016-02-23T06:42:21.489-05:002016-02-23T06:42:21.489-05:00If you are calling the state machine once each tim...If you are calling the state machine once each time through the main loop, then the main loop is where you'd probably put the watchdog. The purpose of the watchdog in this case is to ensure the main loop is alive so the state machine will be able to take a transition when it's time to do so.<br /><br />If you're worried about the state machine doing the right thing that's a different type of problem. You might consider an additional software check to make sure a transition in the state machine happens once in a while (basically a software watchdog), but that's beyond the core watchdog function of making sure tasks are alive and becomes application-specific in a hurry.<br /><br />If you're worried about power down then that also becomes application and chip specific quickly (some watchdogs are used to wake the chip up once in a while -- which is arguably not a watchdog function any more). So that sort of thing is beyond scope for this posting.Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-88595069473249855492016-02-23T05:38:54.920-05:002016-02-23T05:38:54.920-05:00I am working on a system used to controll and mech...I am working on a system used to controll and mechanical clock mechanism, which implies that everything happens really really slowly compared to the uC. My code is designed as classical state machine. I want to use watchdog timer but I am not quite sure how to and where to add feeding the dog. I suppose that main loop is bad place for that? Could you please give some words on that?vlasinachttps://www.blogger.com/profile/16565875046035215220noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-65995891559377816632015-11-07T19:13:43.130-05:002015-11-07T19:13:43.130-05:00This blog material was originally written in the c...This blog material was originally written in the context of safety-critical systems.<br /><br />I can't remember a design review I've ever done where a stalled program was an acceptable outcome. Yes, incorrect watchdog use is a bad thing, but it is not all that difficult to use it correctly. If your system is simply not critical and you're really worried about false alarms, I'd rather see a really generous watchdog interval than see it turned off. (How to determine the interval is a topic for another blog post at some point.)<br /><br />I really disagree very strongly with your 99.5% number, but I'm putting up the post since it seems to based on your experience, which apparently is a lot different than mine.<br /><br />Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-76572836388075835012015-11-07T18:55:06.277-05:002015-11-07T18:55:06.277-05:00In my opinion many software engineers introduce a ...In my opinion many software engineers introduce a watchdog mechanism too light-hearted. You have to ask yourself the question 'What's worse, a stalled program or a reasonable chance my program will randomly reset because of an unexpected delay somewhere?'<br />And in many programs the watchdog is implemented wrong, giving a false sense of security when the watchdog is kicked way too often.<br /><br />Don't get me wrong, I see good reasons for implementing a watchdog but on several occasions I've seen the watchdog code to introduce hard to find bugs. <br />99.5% of all embedded applications can do without a watchdog. Only when no human is present to reset the device on stalling or when lives are on stake a watchdog is required. In all other situations the chance of the watchdog code creating more problems than it solves makes me stay away from this feature when I can.Unknownhttps://www.blogger.com/profile/13570713659384772358noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-11642657960904284312015-03-07T20:12:47.265-05:002015-03-07T20:12:47.265-05:00I appreciate the sentiment you express. Random r...I appreciate the sentiment you express. Random resets are unacceptable. But that doesn't mean you shouldn't have a watchdog. I prefer a philosophy that you use a watchdog, but if the watchdog ever trips (even once) that is a full-on, catastrophic failure of the engineering process, because it should never happen if your code is of good quality and your system is well designed. (I.e., a watchdog trip is a sign of a software fault, not a "business as usual" event)<br /><br />Even with a perfectly designed piece of software you can get a watchdog trip due to a single event upset in the CPU logic. More likely a big software project is less than perfect and there are a few subtle bugs that just happened to slip through. Completely deterministic design reduces the chance of that bug getting past, but sometimes it might still happen. <br /><br />It is interesting to debate whether using a watchdog presents a moral hazard that leads to laziness in coding, but I vote on the side of disciplined design plus watchdog.<br /><br />BTW, a watchdog in fact did save the Mars Pathfinder mission, and I'm sure they are glad they had it turned on:<br />http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/mars_pathfinder.html<br /><br />Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-79929931436938962822015-03-07T18:52:18.262-05:002015-03-07T18:52:18.262-05:00Let me put it to you that Watchdog timers themselv...Let me put it to you that Watchdog timers themselves are bad practice.<br /><br />To accept that a 'random unpredictable reset' is better than a 'un-recoverable halt' is detremental to design and code quality.<br /><br />I believe Watchdog timers give a false sense of security , like a dark cloud hanging over the software engineer form the very beginning of a design. And this leads to sloppy code. Because, no matter what happens 'at least it will reset'.<br /><br />Well. I don't use watchdog timers. Ever.<br /><br />And I am forced to accept that my code must run reliably, or else disaster.<br /><br />What this results in is :<br />Reliable firmware.<br />Because the designer needs to.<br /><br />Ensure that interrupt routines are small, completely predictable, and in-interruptible.<br /><br />The main application is essentially the watchdog. Switching predictably between tasks, polling flags, managing execution.<br /><br /><br />If you run such an application once.<br />You run it a thousand times.<br />Because the execution is independent of the unpredictability introduced by interrupts.<br />And every possible scenario is defined.<br /><br />I am forced to design like this.<br />Because there is no other option.<br />There is no sloppy watchdog ticking away in the background waiting to clean up after my lazy code over runs the stack.<br /><br />A random reset in the middle of a measurement or a mars-rover sample collection, a random reset while controlling the flaps of an aircraft wing. etc.<br />Do not use watchdogs.<br /><br /><br />Watchdogs are a false sense of security.<br />A random reset is unacceptable.<br />Don't allow it into your design.<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-47551319603940198652014-10-12T12:32:16.186-04:002014-10-12T12:32:16.186-04:00Yes, I'm aware of the limitations of a softwar...Yes, I'm aware of the limitations of a software only approach, however there are no safety requirements.<br />Anyway it is an improvement compared to the no watchdog situation since it can catch faults previously undetected, like a single thread like a single thread dead, or "hung".<br />Thanks again for your blog.bashir1961https://www.blogger.com/profile/00627064259877098042noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-23284489372752812092014-09-01T12:28:59.604-04:002014-09-01T12:28:59.604-04:00A software-only watchdog is likely to be better th...A software-only watchdog is likely to be better than no watchdog. But, keep in mind that this won't protect you against faults inside the operating system that cause the watchdog task to die or other common-mode problems caused by running the watchdog and the application on the same platform.Phil Koopmanhttps://www.blogger.com/profile/11849599272360094243noreply@blogger.comtag:blogger.com,1999:blog-4172950626830217643.post-44313139749946134022014-08-27T16:14:26.036-04:002014-08-27T16:14:26.036-04:00Very interesting article, especially on the topic ...Very interesting article, especially on the topic of multiple threads, thank you.<br />I work in a similar field, i.e. pc embedded in automation machines, running windows embedded, console application, no user interface.<br /><br />My plan is to implement a software only watchdog timer, using the techniques you explained, since in windows processes are isolated from each other I think it could be viable.<br /><br />The watchdog application must be of course more reliable than the controlled apps, my plan is to adopt a strict abstract data type implementation, like explained in "C interfaces and implementation", David Hanson, and use a subset of MISRA-C 2004.<br /><br />Since the estimated size of the program should be relatively small I hope to achieve a good code coverage testing, and also to use it as a sort of template for explaining good programming style and techniques.<br /><br />If you think it could be interesting for your blog please let me know, my interest is to grow professionally and make myself known to other professionals.<br /><br />bashir1961https://www.blogger.com/profile/00627064259877098042noreply@blogger.com