Of all the projects I have worked on a big regret has been; adding complexity where it was not needed. I was recently reminded of, how easy it is to fall into that trap.
We are in the process of developing a new feature for a brand new system. As part of the feature we need to run a map reduce query on a data grid. We discovered that we could optimise the query if we maintained an in-memory cache.
The cache was pretty simple - backed by a a couple of hash maps. This part of the system is multithreaded and that is where we saw a possible race condition. So before we added synchronisation we wanted to force the race condition from our tests.
Reliably creating a race condition is obviously very tricky. We looked at the excellent MultithreadedTC library but in this case it was not suitable. In the end we created a test from scratch that, although complicated, did the job. We added read/write locking to make the test pass. And we were done! We could feel good about ourselves.
Hang on! We just added a a lot of complexity but for what? We made an assumption that the query will need optimisation. But does it really? We do not know until we have a performance environment to test the query with expected volumes. This is why we create premature optimisations because we shy away from creating a failing performance test before creating the optimisation. Often we do not even have a performance environment that is a good reflection of production and even if we do we are not always comfortable with writing performance tests. So we optimise just in case adding complexity where it is probably not required.
In our case we have removed this 'premature optimisation'. It is amazing how much code we manage to remove! We have communicated with our sponsor that there is a risk in this feature not performing under peak conditions and we need to create a performance environment along with the tests to understand if optimisation is needed. We also communicated that this performance environment will be required for subsequent features and forms an important part of our testing landscape. It is now on our backlog as a high priority.
Mash is a pragmatic software craftsman always looking to improve his software creation skills and helping others do the same. He firmly believes that a well-rounded software craftsman must have a keen interest in all aspects of software creation, including; process, people, technology, user experience, development, operation, maintenance, and social impact. He relishes the daily challenges that Codurance brings to him–stretching his existing knowledge and expertise allowing him to constantly grow as a professional.
Mash is an advisor and a leader. During his diverse career, he has succeeded in invigorating large ailing software projects as well as creating highly effective software teams and departments. His broad and deep technical knowledge, organisational skills, craft focus, and empathy to people involved have been integral to his success. He has worked in many roles for charities, investment banks, consultancies, government, media and cloud providers. He prides himself at being a hands-on software developer and believes that software development skills are very hard to learn and the best way to maintain them is to apply them.All author posts
Software is our passion.
We are software craftspeople. We build well-crafted software for our clients, we help developers to get better at their craft through training, coaching and mentoring, and we help companies get better at delivering software.