Jekyll2023-11-01T14:44:01+00:00https://higher-order-logic.github.io/feed.xmlHigher Order LogicConcerning software development and other topics.Bad code isn’t Technical Debt, it’s an unhedged Call Option2023-10-06T00:00:00+00:002023-10-06T00:00:00+00:00https://higher-order-logic.github.io/programming/2023/10/06/bad-code-isnt-technical-debt-its-an-unhedged-call-option<p>This is all <a href="https://papachrismatts.uk">Chris Matts</a> idea. He realised that the problem with the “Technical Debt” metaphor is that for managers debt can be a good thing. Executives can be required to take on more debt because it makes the finances work better, it might even be encouraged by tax breaks. This is not the same debt as your personal credit card. Chris came up with a better metaphor, the <em>Call Option</em>.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/thumb/5/5e/Short_call_option.svg/200px-Short_call_option.svg.png" alt=""Payoff from writing a call."" /></p>
<p>I “write” a <a href=""http://en.wikipedia.org/wiki/Call_option">Call Option</a> when I sell someone the right, but not the obligation, to buy in the future an agreed quantity of something at an price that is fixed now. So, for a payment now, I agree to sell you 10,000 chocolate santas<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> at 56 pence each, at any time up to 10th December. You’re prepared to pay the premium because you want to know that you’ll have santas in your stores at a price you can sell.</p>
<p>From my side, if the price of the santas stays low, I get to keep your payment and I’m ahead. But, I also run the risk of having to provide these santas when the price has rocketed to 72 pence. I can protect myself by making arrangements with another party to acquire them at 56 pence or less, or by actually having them in stock. <em>Or</em> I can take a chance and just collect the premium. This is called an unhedged, or <a href="http://en.wikipedia.org/wiki/Naked_call">“Naked” Call</a>. In the financial world this is risky because it has unlimited downside, I have to supply the santas <em>whatever</em> they cost me to provide.</p>
<p>Call options are a better model than debt for cruddy code (without tests) because they capture the unpredictability of what we do. If I pop in an a feature without cleaning up then I get the benefit immediately, I collect the premium. If I never see that code again, then I’m ahead and, in retrospect, it would have been foolish to have spent time cleaning it up.</p>
<p>On the other hand, if a radical new feature comes in that I have to do, all those quick fixes suddenly become very expensive to work with. Examples I’ve seen are a big new client that requires a port to a different platform, or a new regulatory requirement that needs a new report. I get equivalent problems if there’s a failure I have to interpret and fix just before a deadline, or the team members turn over completely and no-one remembers the tacit knowledge that helps the code make sense. The market has moved away from where I thought it was going to be and my option has been called.</p>
<p>Even if it is more expensive to do things cleanly (and I’m not convinced of that beyond a two-week horizon), it’s also less risky. A messy system is full of unhedged calls, each of which can cost an unpredictable amount should they ever be exercised. We’ve all seen what this can do in the financial markets, and the scary thing is that failure, if it comes, can be sudden—everything is fine until it isn’t. I’ve seen a few systems which are just too hard to change to keep up with the competition and the owners are in real trouble.</p>
<p>To me, that makes refactoring like buying an option too. I pay a premium now so that I have more choices about where I might take the code later. This is a mundane and obvious activity in many aspects of business—although not, it seems, software development. I don’t need to spend this money if I know exactly what will happen, if I have perfect knowledge of the relevant parts of the future, but I don’t recall when I last saw this happen.</p>
<p>So, the next time you have to deal with implausible delivery dates, don’t talk about Technical Debt. Debt is predictable and can be managed, it’s just another tool. Try talking about an Unhedged Call. Now all we need is a way to price Code Smells.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>There is an apocryphal story about a trader buying chocolate santa futures and forgetting to sell them on. Eventually a truckload of santas turned up at the Wall Street headquarters of the bank. There’s also <a href="https://thedailywtf.com/articles/Special-Delivery">this story</a>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>This is all Chris Matts idea. He realised that the problem with the “Technical Debt” metaphor is that for managers debt can be a good thing. Executives can be required to take on more debt because it makes the finances work better, it might even be encouraged by tax breaks. This is not the same debt as your personal credit card. Chris came up with a better metaphor, the Call Option.A response to James Shore’s Nullable pattern2023-01-06T00:00:00+00:002023-01-06T00:00:00+00:00https://higher-order-logic.github.io/programming/2023/01/06/response-to-james-shore<p>I’ve been reading James Shore’s pattern language,
<a href="https://www.jamesshore.com/v2/projects/testing-without-mocks/testing-without-mocks">Testing without Mocks</a> with interest.
It has some interesting ideas and, for me, some points of disagreement.</p>
<p>My first objection is to the “clickbaity” title, as if mocks were
some catastrophic evil, rather than one tool amongst many. An interesting
pattern language has to be more than an opposition to a minor technique.</p>
<p>I also see some confusion between concept and tooling. James has since written
another post that clarifies his wariness about
<a href="https://www.jamesshore.com/v2/blog/2023/the-problem-with-dependency-injection-frameworks">Dependency Injection frameworks</a>
but has not yet made the same point about mocking frameworks.
And many of the popular frameworks are not ones I would chose. Unfortunately, almost our
entire modern stack is based on widely-adopted but flawed implementations–
the Unix API being a fine example–so we have to exercise our design skills.</p>
<p>Once again, I have to take on the argument that testing interactions
means testing implementation. There’s a fundamental disagreement here.
To me, for some objects, the calls they make to their surrounding environment,
in response to calls they receive, <em>is</em> the interesting behaviour I want to test
not a detail of implementation.</p>
<p>This is not the only alternative, but there <em>is</em> an established
approach to object design that focusses on the messages they send to each other,
dating back to Smalltalk and Erlang. Ralph Johnson called this the “mystical” school
of objects. There are other schools of object design where this is less interesting.</p>
<p>It is perfectly possible to write tests that express these interactions clearly,
just as it is possible to make a complete mess of them. Most of the bad testing
I see comes from not understanding the concept and, in particular,
testing at too fine a grain.</p>
<p>The most confusing part to me is that the proposed techniques,
combining Nullable objects with Configurable Responses and Output Tracking
sounds exactly like what I do with mocks, except perhaps with the substitutions
plugged into different places.</p>
<p>So, working through the post, these are my reactions to most of his patterns:</p>
<h3 id="state-based-tests">State-Based Tests</h3>
<p>The state vs interaction example that James uses is not one I would recommend. First,
the calls that it overwrites are static, rather than methods on an object that is
passed in, so it’s not forcing the dependency. Second, we have a heuristic of
“Stub queries, expect actions”, so I would not set expectations on simple queries;
by about the third test, I would have factored these out into common setup. Third,
<code class="language-plaintext highlighter-rouge">Moon</code> looks like the sort of object that I would be using in the test directly,
not stubbing out. Given the amount of <em>Reflectomagic</em> involved, I’m not surprised
that these tests run so slowly.</p>
<p>And, by the way, did anyone notice the Feature Envy?</p>
<p>James’ point about localising tests is important. I see one way of coping with scale
it that each level of code tests its own paths, assuming that detail will be
tested further down the line.</p>
<h3 id="zero-impact-instantiation">Zero-Impact Instantiation</h3>
<p>I absolutely agree that constructors should be simple. My approach is that the
only thing a constructor should do is to <em>construct</em> the object, that is assign
its fields. If I have more work to do to get there, then I write a factory
method or function. I find that this ensures that objects are easy to create
in situations I hadn’t planned for, like new tests. This is a technique I learned
years ago from Modula-3 (which I still miss).</p>
<p>If I can, I avoid <code class="language-plaintext highlighter-rouge">connect()</code> or <code class="language-plaintext highlighter-rouge">start()</code> methods on an object,
preferring to do that work outside the object and only constructing
it if I already have a working connection. Of course, that depends on
the runtime lifecycle.</p>
<h3 id="parameterless-instantiation">Parameterless Instantiation</h3>
<p>For code of any size, I often end up building common infrastructure for creating
test objects, using helper methods with descriptive names, so that the tests code
describes just what is immediately relevant. Sometimes I can default out all the
parameters, sometimes I need to make a choice first. The pushback from the
constructor interface is valuable, it shows me in the code how objects are connected.</p>
<h3 id="signature-shielding">Signature Shielding</h3>
<p>Very much agree with this. Test code needs to be treated with the same care and attention
as production code, possible even more because it bridges the domain and testing. I see
many codebases where the reason the tests are brittle is because they’ve never been
refactored. One caveat is that sometimes I allow a bit more duplication in tests to
keep them more readable.</p>
<h3 id="a-frame-architecture-logic-sandwich-traffic-cop">A-Frame Architecture, Logic Sandwich, Traffic Cop</h3>
<p>All good points. Quite often I end up with a higher-level directory that contains
the core concepts of the code, as interfaces and values, with lower-level directories
for various aspects of the implementation.</p>
<h3 id="grow-evolutionary-seeds">Grow Evolutionary Seeds</h3>
<p>I think we’re less far apart than James presents. The flow he describes is similar to
how an outside-in approach would start, especially the hard-coding of values. One
advantage for me of outside-in is that it helps me to think about and discuss what
we want to achieve next <em>in domain terms</em>. Much of the rest of this process is familiar.</p>
<h3 id="early-visible-behaviour">Early-Visible Behaviour</h3>
<p>Agree on most of this, except for providing a getter just for testing, to me that’s
exposing implementation because it locks in some aspect of the object’s internal
representation. The alternative, exposing an event to be listened for, sounds like
a Mock expectation.</p>
<h3 id="testable-libraries">Testable libraries</h3>
<p>Largely agree here. This is why Ports-and-Adapters is not just for services as a whole
but is a useful mid-level pattern within a codebase.</p>
<h3 id="collaborator-based-isolation">Collaborator-Based Isolation</h3>
<p>This is an interesting idea. I tend to build up a collection of representative constants
that I use as markers in tests and their setup, so it would be <code class="language-plaintext highlighter-rouge">INVENTORY_ADDRESS</code> rather than
<code class="language-plaintext highlighter-rouge">123 Main Street</code>. In our approach to the builder pattern, we provide defaults for
common values so that the didn’t need to be specified repeatedly.</p>
<h3 id="infrastructure-wrappers">Infrastructure Wrappers</h3>
<p>Again agree. What’s not made explicit is that I use Interface Discovery to pull in just the features
my code needs from its environment, rather than providing a generic clean veneer over the external
service. Quite often, after a bit of refactoring, I end up with an external service translator that
implements a small set of interfaces driven by the needs of various bits of the main code.</p>
<h3 id="narrow-integration-tests-paranoic-telemetry">Narrow Integration Tests, Paranoic Telemetry</h3>
<p>Yes and yes</p>
<h3 id="nullables">Nullables</h3>
<p>This is an interesting idea. My first question is whether, if one has a version of an object which
behaves partially differently depending on circumstances, is whether this is actually more than one object
and that the nullable part should be factored out.</p>
<h3 id="embedded-stub">Embedded Stub</h3>
<p>While I absolutely agree with keeping things minimal, I struggle with this one.</p>
<p>First, if I need to put it close to the production code to remind me to keep up to date,
then surely I’m missing some tests somewhere? Second, a constant complaint about mocking third-party code is the
difficulty of ensuring that it’s compliant, which is why we recommend against doing that. How does one keep
third-party code stubs compliant in this approach?</p>
<h3 id="configurable-responses">Configurable Responses</h3>
<p>Yes. This is exactly how we started with our fist mock testing, then we got bored of the duplication
and wrote some supporting libraries. We kept it as simple as we could. I struggle to understand why
basic libraries are overkill for stubbing but not for, say, string operations or, even, unit testing.</p>
<h3 id="output-tracking">Output Tracking</h3>
<p>As mentioned above, this sounds remarkably like mock expectations, with the disadvantage that one
can’t trip the debugger with all the state in place when something goes wrong.</p>
<p>I wrote a chapter about using the need for event listening to guide collaborator discovery in
the GOOS book. There was a time when introducing a listener for event reporting to
make testing easier, as against using the log, was a radical concept.</p>
<h3 id="fake-it-once-you-make-it">Fake It Once You Make It</h3>
<p>Again good stuff here. I have a question around the embedded use of Nullables. I like to use the back
pressure of seeing too many parameters being passed in as a hint for missing constructs. If those
instances are created as default nullables, rather than being declared as an explicit dependency, then
I can’t see that.</p>
<h3 id="replace-mocks-with-nullables">Replace Mocks with Nullables</h3>
<p>I’m seeing most of this as a shift in terminology, rather than a meaningful redesign. Maybe that’s
useful enough. One thing I can guarantee is that if this pattern really takes off, then it will be
abused, and then trashed for that abuse, in ways that James never thought possible.</p>I’ve been reading James Shore’s pattern language, Testing without Mocks with interest. It has some interesting ideas and, for me, some points of disagreement.An unreasonable dislike of mutable variables2023-01-04T00:00:00+00:002023-01-04T00:00:00+00:00https://higher-order-logic.github.io/programming/2023/01/04/mutable-variables<p>I’ve had an adverse reaction to mutable variables for some time.
Recently I found another justification for this paranoia.</p>
<p>I was working on a codebase that included a snippet that looked like this:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">all_the_stuff</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">if</span> <span class="n">one_thing</span><span class="p">:</span>
<span class="n">all_the_stuff</span> <span class="o">=</span> <span class="n">search_for_one_thing</span><span class="p">()</span>
<span class="k">elif</span> <span class="n">another_thing</span><span class="p">:</span>
<span class="n">all_the_stuff</span> <span class="o">=</span> <span class="n">search_for_another_thing</span><span class="p">()</span>
<span class="n">do_something_with</span><span class="p">(</span><span class="n">all_the_stuff</span><span class="p">)</span>
</code></pre></div></div>
<p>The build also included a test coverage check, and this section showed as fully covered.</p>
<p>While experimenting with some refactoring, I replaced the variable with a function.
My motivation for this is to simplify the “narrative” at the top level of the code,
pushing the detail into a supporting level, and I use mutable variables as a hint
for where I might do that.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">do_something_with</span><span class="p">(</span><span class="n">all_the_stuff</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">all_the_stuff</span><span class="p">():</span>
<span class="k">if</span> <span class="n">one_thing</span><span class="p">:</span>
<span class="k">return</span> <span class="n">search_for_one_thing</span><span class="p">()</span>
<span class="k">if</span> <span class="n">another_thing</span><span class="p">:</span>
<span class="k">return</span> <span class="n">search_for_another_thing</span><span class="p">()</span>
<span class="k">return</span> <span class="p">[]</span>
</code></pre></div></div>
<p>This passed the tests but failed the build because I had regressed the level of test coverage.
It turned out that the line:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">return</span> <span class="p">[]</span>
</code></pre></div></div>
<p>was not covered.</p>
<p>This showed that there was no test for the case when
neither <code class="language-plaintext highlighter-rouge">one_thing</code> nor <code class="language-plaintext highlighter-rouge">another_thing</code> was true. This was impossible to see
in the coverage trace for the previous version.</p>
<p>This gap is a strong hint to take a closer look at the code
to check whether that path is possible and to either write a test,
if it is, or rework the code to make it clear that it’s not.</p>I’ve had an adverse reaction to mutable variables for some time. Recently I found another justification for this paranoia.