I am sitting here, my work is done and as I am looking out of my window, it is already dark outside and snowing. I live next to the mountains in the Alps in the south of Germany near to the border to Austria. This is the most beautiful time to brew a good cup of coffee and read a Twitter-conversation, I wanted to read the whole day.

tl;dr This is the discussion of the Twitter-Thread, started by Mike Coon, augmented with definitions, background, and some thoughts:

It started with Mike Coon on the 18th of December 2018 and evolved into a long-running, professional conversation with respect and many different topics and attendees. As Twitter is a great medium, supporting the start of these conversations, it is not the best to read them and get some background.

Here is the result of my editing. When reading this, please keep in mind, I am a professional programmer, not an editor. Therefore it is possible if not likely, that I made some mistakes. I am very happy to correct them.

I added my personal comments. In the past, I did this type of work just for my own, but recently I decided to do more in the public, so here we are.

The Beginning

Everything started on the 18th of December 2018, with Mike Coon.

Mike Coon: So there’s a thread about how tech bloggers/speakers don’t use citations compared to say law and academia. Several people seem offended that much of what’s written is ‘opinion’ as if law was not exclusively about opinions.

In the law there are thousands of years of history around the opinions and consequences of those opinions. There are entire structures around exploring those opinions and coming to consensus about various aspects of human behavior.

Still, according to lawyers, when sued for malpractice, the law is not ‘settled’ in many areas.

Similarly, we in the greater space of software development and computer operations have tons of opinions on the best way, right way, ethical way to go about our business and still don’t benefit from having codified those things in a way we can reference and adjudicate.

Should we consider running trials? Put up cases in an adversarial setting with a jury of skilled peers and start creating citable precedent? Could we have advocates for a practice or technique put forth their evidence and the court render an verdict?

Imagine putting TDD on trial and then, if TDD is found to be the correct approach — insisting that professional software devs practice TDD. Adding mastery of TDD to the professional certification requirements to be admitted to the Professional Association of Software Creators

Creating a code of software development statutes… How far would this extend into practices like Scrum or Kanban or code refactoring? Would choosing on-prem servers over serverless cloud services become an actionable offense?

Could we run people out of the business for stacking in-memory databases on top of interfaces that eventually expose the data in an Oracle database — because performance. That might be worth the trouble.

Would creating consensus and codifying practices help move the profession forward or simply discourage further innovation? Would corporate interests find a way to manipulate the system for their gain at our expense?

Mike Coon: Would love to see some discussion around this. Fin

“I am a Symmathecist — teams and organizations. Interested in D&I, Strategy, Agility, DevOps, Code as Craft and a better industry and society. cishet he/him” — Mike Coon on his Twitter Profile

What I understood: Coon recognized, that programmers (speakers/bloggers) don’t user citation, wit the argument, that they present their opinion. He claims, that law is just a set of opinions, proven and matured by a long history and refined again and again, written down and agreed on. In software development, we have a lot of opinions, too, but not “codified those things in a way we can reference and adjudicate”. He invites to imagine, what would happen if we would create our own software development statutes as it happened in other disciplines like law or academia. He opens the discussion with two questions.

Would creating consensus and codifying practices help move the profession forward or simply discourage further innovation?

Would corporate interests find a way to manipulate the system for their gain at our expense?

My Opinion: The discussion diverges quickly off topic. It was not moderated, so I guess this happens easily. The questions are still interesting to think of and the discussion still an excellent read!

Dan Creswell: It’s a reasonable metaphor for comparison equally, the refinement done via case law is to identify the ultimate intent/interpretation of the law. There’s a piecewise refinement if you will.

Certaintly opinions are being traded but it’s against a baseline of law and prior case law (albeit a little imperfect as folks can miss things, a certain amount of winning the argument by virtue of others not doing diligence).

And the process is somewhat defined with this in mind and attempting to neutralise the worst imperfections.

Do we see the same generally in tech? I’m not sure but then blogging and speaking is all about opinion.

And it’s rather different from either standing in a court or writing and defending and peer reviewing of papers. Notably bigger technical firms do contribute in these arenas though that may be a function of money.

What might be a concern IMO is that as an industry we don’t recognise the differences and take care to delineate between opinion, legal process, theory vs practice, scientific method and its limitations etc. We might be treating it all more equally, maybe even unkowingly so.

Software Dynamicist — Tech is the servant of humanity. Dabbler in martial arts. — Dan Creswell on his Twitter Profile

Mike Coon: I think this is a valid criticism from outsiders with more settled structures and systems. Even though they all have sketchy to horrific histories leading to those structures and systems. And struggle mightily in areas of innovation — our lifeblood.

Dan Creswell: ‏Some failings are about the human condition :) If I were being critical, I would say we’re not striving hard enough to find our own structures and systems. If we could generally admit that we’re missing these things and start looking for them, t’would be progress

Of course there will be a class of folks who are disadvantaged by such a move and that cannot be forgotten. I don’t think we have to do it the same way as other industries but we should be striving to find our own equivalents. I don’t feel like I see much of that generally yet.

The resurgence of the conversation

With the last answer, the conversation stopped, but Mike Coon resurged it with a new post.

Mike Coon: Really thought somebody would have some thoughts on this. Maybe @RonJeffries was right and I am just screaming into the void.

I’ll try spamming now.

CC: @GeePawHill @tottinge @TotherAlistair @tastapod @dancres @unclebobmartin @AndyPalmer @mfeathers

Peter Kretzman: I have a lot to say on this. Key issue in my view is that many folks in the tech arena have repeatedly shown themselves unable to distinguish between pure emotional opinion and a reasoned stance. To those folks, everything that isn’t a verifiable fact is pure opinion, all equal.

Veteran IT executive, writer, speaker. Still codes. Author of the blog, ‘CTO/CIO Perspectives: Intensely practical tips on information technology management’ — Peter Kretzman on his Twitter Profile

Alistair Cockburn: A few things are science/facts by now; but still not universally applicable

A lot of things are techniques-fit-to-people, so varying.

I did a brief expert-witness thing in a lawsuit for gross mismanagement of a project. I could show what they did was known to be inefficient

but most things are still “common practice” or “fit-to-invidividual”

Chaotic-good Lawful-neutral project witchdoctor. Bard. Playful, wreaks havoc on the unsuspecting. Dancer, specialty massage, poet. Sits underwater. — Alistair Cockburn on his Twitter Profile

Mike Coon: I’m not sure this is a bad thing — common practice…. Would the field be helped or hurt by capturing those things that are science/facts into a citable canon?

Dan North: Common practice is a nightmare. Cost accounting is common practice. Big-four audits of major corps is common practice. Handing huge govt contracts to failed tech contractors is common practice. As Eli Goldratt said about slow adoption of ToC: “Common sense isn’t common practice.”

Optimizer of organizations, teams and software, programmer, Agile coach, technologist, troublemaker. Christian, husband, occasional blogger. — North about himself on Twitter

Note by the Editor: Eli Goldratt is the author of the novel “The Goal” which is about a plant manager. The DevOps-Book “The Phoenix-Project” refers to this heavily and has many similarities. ToC means “Theory of Constraints” which was developed by Goldratt. ToC the science of the phrase: “a chain is no stronger than its weakest link” applied to workflows.

Mike Coon: So you’re saying it would hurt to agree on some of the science and facts?

My Opinion: This was a little too emotional formulated for me, especially as Coon claimed, but it seemed to help the conversation, so maybe a good technique… I start to see, that the definition-problem of vocabulary start, what we well see later again and again. A big problem in nearly every IT-Discussion. We should maybe create our own definition-register?

Dan North: I’m saying there is no “science” or “facts” in the sense I think you mean them. At best it is a social science (and “any subject with the word ‘science’ in it isn’t”) with all the limitations that implies. TDD, for example, is great in some contexts, counterproductive in others.

Conversation shifts to TDD

So, now is the moment, to think the last time about the original questions:

Would creating consensus and codifying practices help move the profession forward or simply discourage further innovation?

Would corporate interests find a way to manipulate the system for their gain at our expense?

Coon shifts the conversation towards the always hot topic TDD and other persons jump in.

Mike Coon: I haven’t seen a context where TDD was harmful. I can imagine some test first practices that might be terrible though.

Barney Dellar: I sometimes write little scripts that download things, copy them, that kind of thing. I don’t use TDD, and I don’t believe it would help if I did. And there’s a continuum from tiny scripts to massive applications.

At one end of that continuum, TDD isn’t useful, and at the other, it’s a fantastic practice that I would strongly recommend.

Could we use Cynefin or something similar to decide on recommend practice? Like, “TDD is recommended/essential for the Complex domain”?

Symmathecist in the medium of software | C++ | Agile | singer | atheist | socialist | husband of @velvet_selkie | dad | Green | cat-lover | he/him | — Dellar on his Twitter Profile

My Opinion: I can’t agree more on “TDD is recommended/essential for the Complex domain” but sometimes it is often also useful to craft a throwaway code in a “simple domain,” like parsing a file to break the problem in a more biteable one.

On the discussion “Is TDD dead?”, Kent Beck gives also a great example when it makes no sense to use TDD. See here (pos encoded in the link)

Dan North: I would head over to Liz Keogh’s blog and take a look at some of the awesomeness she’s written about. I’m pretty sure exactly this topic comes up. Although I may be conflating it with her talks.

Note from the Editor: Liz Keogh’s blog https://lizkeogh.com/

Mike Coon: Yes. I suspect there’s content from her that I am not finding and really need to read.

Uncle Bob Martin: Anything @lunivore writes is worth reading.

Software Craftsman — Uncle Bob Martin on his Twitter Profile

Liz Keogh: Thanks @unclebobmartin :) This was the article I wrote (and Dan is right, I shouldn’t have called it “Beyond TDD”). Generated a *lot* of comments regardless.

https://lizkeogh.com/2012/06/24/beyond-test-driven-development/

I’d use TDD for anything predictable, and for anything where we’re iterating towards a known outcome (liminally complex/complicated). For everything else, TDD the known bits and spike the rest out. And if in doubt, do TDD; it takes some practice to know when it’s OK not to IMO.

Lean/Agile consultant, speaker, blogger, poet, fantasy author, pattern hunter, complexity thinker, free will skeptic, Londoner, BDDer, still Goth. — Keogh on Twitter

My Opinion: With “in doubt, do TDD; it takes some practice to know when it [TDD] is OK not to IMO,” Keogh already nailed it.

Shift to the discussion on details on “How to do TDD”

Graham Cox: For me, as a Java developer, TDD works fantastic on the outsides of systems — e.g. testing the HTTP layer of an API service. I’ve always struggled with it on the inside though, because I don’t have code to test yet and so it doesn’t compile. Always felt I was missing something…

Ron Jeffries: Not compiling counts as a red test. We write a test for something that doesn’t exist, and make it exist. Then we write a harder test.

assert_equals(0, new BowlingGame().score()); requires creation of the class and method returning 0;

Some folks use a smaller test than that.

I’m sure you can figure out who I am if you really want to. — Ron Jeffries on his Twitter Profile

My Opinion: Well, like a special form of test. Most tests I normally think of, test the semantics of a program. Compiling proofs syntactic correctness and if you have a type of partial correctness regarding the types. But these are proofs and not examples/tests, what becomes a topic later.

Definition (Oxford): Correctness Proof.

Correctness: “The quality or state of being free from error; accuracy.”,

Proof: “Evidence or argument establishing a fact or the truth of a statement.”

Ciaran McNulty: Depending on the tooling, compiler errors aren’t as friendly as red test outputs

Trainer, consultant, developer, coach, scrum master, BDD agitator. Lead maintainer of @PhpSpec. Sometime conference speaker. I pronounce my name KEER-un — McNulty on Twitter

Note: BTW, I am pretty sure, that the new persons in the conversations didn’t read the full conversation.

Ron Jeffries: No, they aren’t, but the rules say that not compiling counts as red.

Ciaran McNulty: I’m just making the point that sometimes people find it hard to treat them as such because of platform constraints

Ron Jeffries: Yes, it’s often difficult. In that case I might try a “TDD As If You Really Mean It” style, putting the new construct directly into the test, per @keithb_b, if the IDE would allow. Or, since I know it’s bound to fail, type in a shell class.

Ron Jeffries: What I find, too often to ignore safely, is that after that first push to start TDDing, I am very often glad I did, and without it, I often regret not doing it when it was relatively easy.

YMMV, of course.

Graham Cox: I find that, when I do put the effort in to do it, I hugely appreciate it. It makes me more certain about the quality of the code and it doing the right thing.

The problem is that, too often, I don’t and end up doing the tests afterwards. And it feels “good enough” but not great.

Ron Jeffries: Yes. Test-after, for me, would require far more discipline, of which I have little or none, and even if I did it, it wouldn’t inform my design as does TDD.

Graham Cox: I know I’m the one who needs convincing here, not you :-) but test-after ends up testing the code you wrote, not proving the the code does the right thing. TDD proves that the code you are about to right correctly solved the problem you’ve got.

Transition towards BDD and ATDD

With the last three tweets, the transition of the conversation towards BDD and ATDD began. Interesting: It begins with the argument of Cox, that test-after brings him additional confidence.

Keiþ Braiþvvaite: If only. The best that TDD can do is evidence that the code does what I said I wanted it to. Test-after can do that too, but, per Ron’s comment, requires much more discipline.

Graham Cox: That’s the point though.

TDD proves that the requirements are filled by the code, because the tests *are* the requirements.

Test-after proves that the code does what it’s coded to do, which is hopefully what the requirements say.

Nether prove that the requirements are correct…

Ron Jeffries: No. You may be thinking of Acceptance Test Driven Development. TDD is a programming thing and only rarely does a TDD test mention things at the requirements level. Maybe the thing you don’t like isn’t the thing you think it is?

Ron Jeffries: My rough defs:

Analysis: the activity of determining what the problem is.

Acceptance tests: Detailed checks showing problem input and checking output.

TDD: a programming technique where the developer writes a micro test and a few minutes later has code passing that micro test.

My Opinion: I like the definitions, therefore here in a more formatted way:

Analysis : the activity of determining what the problem is.

: the activity of determining what the problem is. Acceptance tests : Detailed checks showing problem input and checking output.

: Detailed checks showing problem input and checking output. TDD: a programming technique where the developer writes a micro test and a few minutes later has code passing that micro test.

Dan North claims later: “Sorry, wrong,, to most of the above, especially about BDD and quite a lot about TDD”, what could include this. It is North’s opinion. Agree or not agree…

Graham Cox: Given that then yes, I think I tend to conflate TDD and Acceptance tests into the same thing.

And then what I term Acceptance Tests are basically tests against the whole system, not just one class/component.

Joys of different terms for the same thing :-)

Barney Dellar: Testing the whole system is more BDD (Behaviour Driven Development). And the important part there is to have the conversation with stakeholders to make sure you capture the requirements well.

Ciaran McNulty: ATDD and BDD are largely the same idea (born on different continents)

Note: This is an important (and wrong) statement for the conversa

Ron Jeffries: BDD has grown up. It did start much like ATDD but has grown to include some excellent ideas that I’d classify under “analysis”.

Barney Dellar: BDD is about “building the right thing”.

TDD is about “building the thing right”.

BDD is about “building the right thing”.

TDD is about “building the thing right”.

- Barney Dellar

Uncle Bob Martin: BDD and TDD are both about saying what you will do, and then doing what you said. BDD’s scenarios, and TDD’s micro-tests are both promises of behaviors to be implemented. Microtests are also promises of design constraints and boundaries.

Graham Cox: So, if I interpret that right, BDD is to the system as a whole what TDD is to the individual pieces of code.

Both are written ahead of time, with input from the correct people, to make sure that what you’re building does what you expect it to do.

Dan North: Sorry, wrong,, to most of the above, especially about BDD and quite a lot about TDD.

Note: Dan North is heavily involved with BDD andDan North created a BDD framework, JBehave. Therefore, he has a strong opinion on the topic BDD, where the conversation now shifted to.

Dan North: BDD was a direct reframing of TDD, root and branch, from the outset. It had nothing to do with ATDD for about a year, until @PapaChrisMatts noticed that what I was describing applied to analysis as well.

On Tests, Checks, and Examples

With the next post, the definition-problem on checks, tests, and examples starts. I will provide the definitions of the Oxford Dictionary, but use your own head! This is a conversation between experts and not a bible!

Dan North: If I had a time capsule I would go back and call it “Example-Guided Design”. TDD is a terrible name for a powerful design technique. BDD is a less terrible name. They both describe the same process, at both the code and feature level.

You start with an example, which TDD incorrectly calls a failing test, and then w you write some code to get the example to work. You don’t have a “passing teat” except in the most trivial, one-dimensional sense, but you do have a working example.

Definition (Oxford) Example: “A thing characteristic of its kind or illustrating a general rule.”

My Opinion: I like the term “Example Guided Design” or even “Example Driven Development,” because you search for concrete examples and try to generalize them to real software. But afterward, the examples become tests, because you actually test your software via samples (the examples from before).

Charlie Poole: This: “which TDD incorrectly calls a failing test” is what kept me from paying sufficient attention to BDD for several years. That was my kids but I blame it on those who started out with such a combative attitude towards TDD.

The result was that BDD was very attractive to those who had failed to understand TDD in the first place. Those of us who had learned TDD as a way of coding by example were often put off by claims that TDD was “wrong” and BDD “right”. Unfortunate.

Developer, Coach, NUnit guy — Pooles Twitter

My Opinion: Poole has a good point, that many devs, that failed with TDD and saw a new opportunity in BDD. They knew more at this moment because they had (per definition) more experience with the whole automated testing thing. IMHO it is a weakness of humans in common, that they tend to not be reflective and prefer to defend their position without thinking of the possibility, that they did something wrong in the past. This leads to many many productive discussions because no one want’s to understand the other side. I am probably such person as well (hopefully just from time to time).

Chris Matts: The “BDD is TDD done right.” was by one of the TDD crowd. It kinda signalled the end of the fighting in c2006/2007 cc J. B. Rainsberger

Laziness is an aspiration that few achieve. — Matts on Twitter

Note: I don’t know what c2006/2007 is, but it could be The worlds shortest article on BDD. I found it linked under the article “BDD IS LIKE TDD IF…” from Dan North, where you can find very very good comments.

Charlie Poole: Was it? That’s interesting… I always took it to mean that anyone doing TDD was doing it wrong. Of course both TDD and BDD are visions, not actions. Either can be done right. Either can be done wrong.

Definition (Oxford) Vision: “The ability to think about or plan the future with imagination or wisdom.”

Definition (Oxford) Action: “The fact or process of doing something, typically to achieve an aim.”

Dan North: What does “visions not actions” even mean? Both describe a specific way of doing things, both are techniques, any technique can be done well or poorly or incorrectly. Neither is a “vision”.

My Opinion: I agree with North. All these are practices and way to concrete to be visions. But then I think Poole was focused on “plan the future.” TDD/BDD are ways to get concrete with the vision in our heads.

CharliePoole: Visions in the sense that we hope people will perform certain actions and do it well. Actions in the sense of what folks actually do on a project.

Tim “Agile Otter” Ottinger: Curiosity space is the difference between intentions/expectations and actually.

It represents a mystery, a story to unravel.

What do you see in this curiosity space, Charlie?

Anzeneer, Industrial Logic. Co-author of AgileInAFlash & Clean Code. Feedback: http://sayat.me/tottinge warning: curious inner child at work. He/Him — Ottinger on Twitter

Definition (Oxford) Curiosity: A strong desire to know or learn something.

CharliePoole: Nice term, which I hadn’t heard before.

I see lots of different behaviors, which are neither TDD nor BDD. Stuff like writing the tests last or defining entire interfaces up front, in teams that say they are doing otherwise.

Tim “Agile Otter” Ottinger: So that’s the actuals that vary from the expecteds.

What is the story as to why they differ?

Why are these the actuals, why did we expect differently, what happened in-between?

CharliePoole: That doesn’t seem to me to be a question that can be answered generally, but only in the context of a specific team. Or am I missing your point?

Liz Keogh: I think we can tell stories about specific teams. Lots of the teams I work with for instance write tests first because they’re learning; I imagine any team where they’ve been told that “That’s not TDD” may fudge the truth about what they’re doing.

CharliePoole: That’s clear. I’m just not seeing the question as it might fit in the context of the thread.

Tim “Agile Otter” Ottinger: Ah. I may be branching. With all the talk of “BDD done wrong/right” and “TDD done wrong/right” the voices in my head are saying “rather than condemning, what have we learned in all of this.”

But I think we have learned it’s not either/or where BDD & TDD are concerned.

Back to Test, Check, and Example

Tim “Agile Otter” Ottinger: Whether we’re settled on whether “test/check” or “example” create a significant semantic difference in training space? I don’t think we’ve learned that yet.

CharliePoole: Well my point was that all the TDD folks I respect talk about examples, which eliminates that as an issue. That leads me to want to elaborate on how I see the word “test” and use it as a coach.

In one sense, “test” might be a terrible word, because it has other uses. For example, although I haven’t yet seen it, programmers might confuse it with a blood test and who knows what they would do! And yet… it doesn’t seem to happen.

Other possibilities are the test for a driver’s license, the final exam in Data Structures 101 and… oh yes… testing as done by testers.

So why isn’t that confusing… at least not to most folks I have worked with? It seems like we humans are optimized to some extent to deal with ambiguity. In programming, we call that overloading.

Ignoring the flippancy, I would say it /is/ confusing, which was my original point (waay back!). Programmers who do “test-driven” design think they have for (software) testing covered. They haven’t.

Ron Jeffries: Maybe some programmers think that. I would think that no programmer who had taken a TDD class, read a TDD book, or thought for a few minutes, could think that.

Chris Cooney: I think the interesting bit here is whether TDD is correct to call it a “failing test” which we don’t seem to have agreed on. People are always gonna do stuff badly — the Q seems to be if TDD *directly encourages* that behaviour with its terminology?

Senior Software Engineer @ Sainsbury’s. My views are boring and often repetitive and nothing to do with my employer. Enjoy. — Cooney on Twitter

What is Correct?

CharliePoole: We haven’t even agreed, it appears, on whether “correct” is a good term to use in a discussion of this type.

Definition (Oxford) Correct: Free from error; in accordance with fact or truth.

Ron Jeffries: You’d have to say “this is incorrect if you use my specific definition of the word”, in order to get even close to accurate.

Chris Cooney: Okay, grab the clippers. Looks like we’ve got some yak shaving to do.

John Bywater: What’s confusing in this thread are the judgements about the correctness or incorrectness of things. Testing demarcates regions in implementation space that are adequate. Refactoring moves you around such a region. That’s the important concept. Correctness is anathema to that.

Python developer — Bywater on Twitter

Dan North: No, testing doesn’t do that. Domain context boundaries do that. Testing does something else.

Chris Cooney: I really don’t want to me that guy but I suspect there’s a misunderstanding around the definition of “adequate” — perhaps rephrase as “tests demarcates regions in implementation space that do what the developer wants them to do”?

Definition (Oxford) Adequate: Satisfactory or acceptable in quality or quantity.

What is a Test?

John Bywater: No, the problem is “test”. A test is something that passes or fails. If the test fails, whatever was under test is deemed to be inadequate with respect to that test. This is more or less the cornerstone of the constructivist or materialist or engineering approach to the world.

Dan North: That’s not true either. A test could have a score or ranking. A test could have a result which was a shade of colour, like a litmus or pH test, or some value. Maybe your understanding and use of “test” is constrained by experiencing it to mean an executable example in TDD.

John Bywater: You’re confusing testing with reporting! https://en.oxforddictionaries.com/definition/test

The question is “does it work?” It doesn’t matter if you are a “tester” or a “developer” or a screenwriter or a scrubber or politician or a priest. We are all handymen, each with his or her little machines, and we have a need to know if they are working.

Definition (Oxford) Test: A procedure intended to establish the quality, performance, or reliability of something, especially before it is taken into widespread use.

Definition (Oxford) report (verb): Give a spoken or written account of something that one has observed, heard, done, or investigated.

Dan North: Sure we do. That isn’t what we are doing when we use examples to guide design though. That is what we are doing when we test the little machines.

John Bywater: Didn’t see this yesterday — some agreement :) but I don’t see what that distinction accomplishes. “You should always write more tests?” An example can function as a test, but only if it can somehow fail… Perhaps whatever can fail can function as a test, and so as an example?

My Opinion: I like “An example can function as a test, but only if it can somehow fail”

Dan North: They can be used as tests once the code exists, sure. And that’s mostly how people use them. But they don’t make nearly as good automated tests as purpose-built automated tests, designed for testing. As I said, you can use a butter knife as a screwdriver.

That’s great in the abstract, but when you are using butter knives — because they are the right thing to use — but calling them “screwdrivers”, and when people start using them as screwdrivers, and worse, think they have good screwdrivers, you are doing screwdrivers a disservice.

Ron Jeffries: What evidence have you that they are not as good as “purpose-built” tests, designed for testing? What evidence have you that anyone anywhere every actually does that kind of testing? And what evidence have you relating to the use of automated /acceptance/ tests in addition?

Filip Milovanović: Late reply, but if we take a test to be any kind of experiment or a measurement of some aspect of the system, isn’t Dan North just saying that TDD tests, while useful, only cover a small subset of all the data points we might be interested in (which we may collect in other ways)

Ron Jeffries: His words say that what we call a “test” in TDD is not in fact a test. I take that to mean that he has some other notion of the word “test” which is somehow more correct than ours.

Dan North: Not quite. I’m saying the code example you write with the purpose of guiding your design, while it can be repurposed as a test after the fact, isn’t nearly as good /at being a test/ as something written with the purpose of testing your software.

Matteo Vaccari: Can you explain how the two would differ? I could I easily spot one kind or the other?

Extreme Programmer, trainer, developer, sometimes agile coach. ThoughtWorker. Esperantist. Christian. — Vaccari on Twitter

Dan North: I was writing a regex to match a type of domain string. I used about 4 or 5 examples to glide a design I was confident about. To actually /test/ the regex I wrote a much more robust and comprehensive test, based on the handful of examples, which I checked in along with the code.

My Opinion: I do this a lot and it works out very good. Even better if you can see a behavior after doing some TDD and add a Property Based Test.

Ron Jeffries: When I TDD, I write the things I call “tests” to drive out another bit of behavior. I specify exactly what it should do, then make it do that. When I’m done doing that, I very rarely write additional tests, because the ones I’ve already done cover all my cases.

If I think of another case that might not work, I write a “test” for it exactly the same way I wrote the others. Each is, yes, an “example” of something that should work. When I refactor to improve the design, I run all the things I call “tests” to be sure they still work.

If they do not work, that tells me that the thing I thought was a refactoring was not, because it changed the system’s behavior. When they do (finally) work, they give me confidence that the thing I have is the thing I set out to build.

My Opinion: I think we have two camps here: Programmers who write additional tests afterward and programmers who try to cover them all before. I am in the first camp, but I don’t claim that one side is right.

Dan North: I agree with all of this. I call this development, using example-guided design. This asserts the positive — the software does what it should do — from a behaviours perspective. What most programmers IME don’t then do is assert the converse — that it doesn’t do what it should not.

They also tend not to test from other perspectives — I use examples of compliance and security but there are many others — once the code “works” based on the examples that guided them. This the examples-as-tests are useful but not sufficient to provide confidence to anyone else.

The End

This is the end of the conversation, that was available as I read it. I really enjoied it. I don’t want to provide a conclusion, as many different professional voices spoke out of experience, that I don’t have. I think everyone should judge by himself/herself.