<\/p>\n\n\n\n
In my opinion, this is one of the most important best practices of all when it comes to maintaining large software frameworks or extensive libraries over the long term. Even though this post is mainly about how to avoid bugs in the first place through targeted testing, you won\u2019t be able to prevent all of them. Despite all your efforts, the following still applies: even with intensive automated testing, you will never end up with zero bugs. Yes, you will have significantly fewer errors, but a few bugs will always slip through. No person and no code is perfect.<\/p>\n\n\n\n
The crucial point is how you deal with these errors. Every time a bug appears, it\u2019s a warning signal: at some point, a test case was missed. Even if the first impulse is to immediately jump into the code and fix the bug, it\u2019s worth pausing for a moment.<\/p>\n\n\n\n
Ask yourselves:<\/p>\n\n\n\n
\ud83d\udca1Mnemonic:<\/strong><\/p>\n\n\n\n Every bug gets its own unit test.<\/strong> When you discover a bug, first write a unit test that fails. Only then fix the bug, and keep the unit test forever to ensure safety during refactoring.<\/p>\n<\/blockquote>\n\n\n\n I can recommend the following procedure:<\/p>\n\n\n\n This approach has two advantages:<\/p>\n\n\n\n I have experienced it many times: years later a problem reappears because someone changes the code and didn\u2019t know that this reactivated an old bug. With a unit test, that would never have happened.<\/p>\n\n\n\n In short:<\/strong> A fixed bug without a test is not a fix, but a pause.<\/p>\n\n\n\n Especially when you work a lot with legacy code or in large frameworks, this approach can make your work considerably easier.<\/p>\n\n\n\n A common mistake: changing the code only to make it easier to test\u2014for example, by adding extra return values or helper variables that provide no added value to the actual function.<\/p>\n\n\n\n That is an anti-pattern.<\/strong><\/p>\n\n\n\n A test should check whether<\/strong> a function reacts correctly and not how<\/strong> it is structured internally. \u27a1\ufe0f The function here returns additional information<\/strong> \u2013 not because the system needs it, but only because the test would like to have it.<\/p>\n\n\n\n This is what the tests then look like (for maximum readability, even for beginners, the use of What\u2019s wrong with it:<\/strong><\/p>\n\n\n\n And the test for it?<\/p>\n\n\n\n \u27a1\ufe0f Clear, simple, correct.<\/strong> \ud83d\udca1Mnemonic:<\/strong><\/p>\n\n\n\n If you have to restructure the code just so the test can \u201cunderstand\u201d it, you\u2019re not testing the behavior\u2014you\u2019re misusing the implementation.<\/p>\n<\/blockquote>\n\n\n\n Tests should not dictate to the code what it has to return, but should only check whether the behavior is correct<\/strong>. But this should not be confused with the idea that your code hardly changes once you start with TDD. On the contrary\u2014it will change significantly over time. It will become more modular, more readable, and much more maintainable. Artificial helper variables just for testing, however, are not part of that.<\/p>\n\n\n\n A good unit test has three characteristics:<\/p>\n\n\n\n \ud83d\udca1Mnemonic:<\/strong><\/p>\n\n\n\n A good unit test is compact, self-contained, and tells you right away whether everything is correct.<\/p>\n<\/blockquote>\n\n\n\n A test name is often the first thing you or a team member see when a test fails. Names like Better:<\/strong> The test name tells you at a glance:<\/p>\n\n\n\n A possible and often practical naming scheme in TwinCAT\/TcUnit is: But that is only one<\/strong> variant. Depending on the team and project, other patterns may also work well\u2014the key is that the name is clear, unambiguous, and self-explanatory.<\/p>\n\n\n\n \ud83d\udca1Mnemonic:<\/strong><\/p>\n\n\n\n A good test name is like a good commit message: you immediately understand what it\u2019s about\u2014without any further comment.<\/p>\n<\/blockquote>\n\n\n\n One of the biggest mistakes I keep seeing in teams: waiting with automated tests until the perfect CI\/CD pipeline is in place. Everything is supposed to be highly professional from the start\u2014with Jenkins, automatic reporting, merge checks, notifications, and all the bells and whistles. Sounds great, but in practice it often leads to nothing happening at all. Perfection can lead to stagnation (see the blog post Perfektionismus als Gift<\/a>).\n\nThe path is much simpler: start immediately with automated unit tests, even if you still trigger the pipeline manually at the beginning. This has nothing to do with \u201cmanual testing\u201d\u2014the tests themselves, of course, run fully automatically. Whether you press the start button or a PR hook does it is completely irrelevant at first.\n\nWhy this is so important is shown by a simple calculation: let\u2019s take a library with 100 unit tests. If these tests don\u2019t run automatically but have to be started individually, and each test including setup takes 5 minutes, you end up with 500 minutes\u2014more than eight hours\u2014for just one test run. Impossible in daily work.\n\nBut if the tests run automatically one after the other, the manual effort is practically zero. You just see at the end whether everything is green. The rest of the perfection\u2014e.g., having the pipeline run automatically on every merge\u2014you can add later.\n\nThe crucial thing: even this first step already saves you 99% of the effort. You don\u2019t need to wait until everything is perfect to get the biggest part of the benefit.<\/p>\n\n\n\n \ud83d\udca1Mnemonic:<\/strong><\/p>\n\n\n\n Better to start with automated unit tests today and trigger the pipeline manually than to wait months for the perfect automation.<\/p>\n<\/blockquote>\n\n\n\n <\/p>","protected":false},"excerpt":{"rendered":" Jeder Bug, sein Unit-Test Meiner Meinung nach ist das hier eine der wichtigsten Best Practices \u00fcberhaupt, wenn man gro\u00dfe Softwareframeworks oder umfangreiche Libraries langfristig warten m\u00f6chte.Trotz aller M\u00fche gilt: Selbst mit intensivem, automatisiertem Testen werdet ihr nie bei null Bugs landen. Ja, ihr werdet deutlich weniger Fehler haben, aber ein paar Bugs werden immer durchrutschen. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[7,40,14],"class_list":["post-365","post","type-post","status-publish","format-standard","hentry","category-wirtschaft","tag-autolab","tag-best-practises","tag-software-development"],"_links":{"self":[{"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/posts\/365","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/comments?post=365"}],"version-history":[{"count":3,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/posts\/365\/revisions"}],"predecessor-version":[{"id":368,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/posts\/365\/revisions\/368"}],"wp:attachment":[{"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/media?parent=365"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/categories?post=365"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tbenninger.com\/en\/wp-json\/wp\/v2\/tags?post=365"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
\n
Test behavior<\/strong>, not implementation<\/strong><\/h2>\n\n\n\n
And under no circumstances should the test force the code to \u201creturn\u201d something that it does not actually need<\/strong>.<\/p>\n\n\n\n
\n\n\n\nAnti-pattern: Code is twisted \u2013 just for the test<\/h3>\n\n\n\n
def cancellation_allowed_bad_practise(status):\n if status == \"delivered\":\n return False, \"already delivered\" # second return value only for testing purpose\n elif status == \"open\":\n return True, None\n elif status == \"payed\":\n return True, None\n return False, \"unknown status\"<\/code><\/pre>\n\n\n\n@pytest.mark.parametrize<\/code> was deliberately avoided):<\/p>\n\n\n\ndef test_cancellation_permission_with_justification():\n allowed, reason = allow_cancellation_bad_practice(\"delivered\")\n assert allowed is False\n assert reason == \"already delivered\"\n allowed, reason = allow_cancellation_bad_practice(\"open\")\n assert allowed is True\n assert reason == None\n allowed, reason = allow_cancellation_bad_practice(\"bezahlt\")\n assert allowed is True\n assert reason == None<\/code><\/pre>\n\n\n\n\n
True<\/code>\/False<\/code> return value would be sufficient.<\/li>\n\n\n\n
\n\n\n\nBetter: function does exactly what it is supposed to do<\/h3>\n\n\n\n
def allow_cancellation(status):\n return status in [\"open\", \"payed\"]<\/code><\/pre>\n\n\n\ndef test_cancellation_permission():\n assert cancellation_allowed(\"delivered\") is False\n assert cancellation_allowed(\"open\") is True\n assert cancellation_allowed(\"payed\") is True<\/code><\/pre>\n\n\n\n
The test asks: \u201cIs cancellation allowed?\u201d<\/em> \u2013 not: \u201cWhy exactly not?\u201d<\/em><\/p>\n\n\n\n
\n\n\n\n\n
Keep tests small, independent, and clear<\/h2>\n\n\n\n
\n
\n
Use meaningful test names<\/h2>\n\n\n\n
It is your first\u2014and sometimes only\u2014documentation of what this test is actually checking.<\/p>\n\n\n\nTestCase17<\/code> or FB_Test_03<\/code> are nothing more than riddles. They force you to open the code just to find out what it\u2019s even about. That wastes time unnecessarily and makes debugging harder.<\/p>\n\n\n\n\n
FB___When<\/code><\/p>\n\n\n\n
\n\n\n\n\u274c Bad names (uninformative)<\/h3>\n\n\n\n
\n
TestCase17<\/code><\/li>\n\n\n\nFB_Test_03<\/code><\/li>\n\n\n\nCheckStatus<\/code><\/li>\n\n\n\nRun1<\/code><\/li>\n<\/ul>\n\n\n\n
\n\n\n\n\u2705 Good names (clear and descriptive)<\/h3>\n\n\n\n
\n
FB_ConveyorControl_StartsMotor_WhenStartCommandIsTrue<\/code><\/li>\n\n\n\nFB_SafetyDoor_Locks_WhenMachineIsRunning<\/code><\/li>\n\n\n\nFB_TemperatureControl_ShutsDownHeating_AboveMaxTemperature<\/code><\/li>\n\n\n\nFB_RobotAxis_MovesToHomePosition_WhenResetCommandIssued<\/code><\/li>\n\n\n\nFB_EventManager_LogsError_WhenInvalidEventReceived<\/code><\/li>\n<\/ul>\n\n\n\n
\n\n\n\n\n
Better to start than to wait<\/h2>\n\n\n\n
\n