Acceptance testing as a production necessity

All programmers are optimists. Perhaps, this modern kind of witchcraft is especially attractive for those who believe in happy endings and good fairies. Perhaps hundreds of failures repel everyone except those who are used to focusing on the ultimate goal. Or maybe it’s just that computers and programmers are young and optimism is inherent in youth. Anyway, the result is one thing: “This time it will definitely go! Or: “I have just detected the last mistake! , F.Brooks

I can’t count how many times my colleagues and I have stumbled upon the problem Brooks described. Recently, there has been a useful habit of stopping and thinking about what you’re doing. It’s not like throwing a checker at the top of the list. Probably 7 years in software development push on new schemes of perception of problems. I want this post to be a beacon, which will make you think about how to do it right at the right time, without rush.

I chose 3 practical tasks that each developer was likely to face. Many of the developers did these tasks and were as lazy as I was, to first come up with a way to measure the success of the task. Before I start, I have to answer the question: “How can I know that I did everything right? About everything in order.

  • DB migration
    Everyone’s been migrating from the old database to the new one. As usual, this process is built in an ideal world? We study the database schema, the connections in the tables, write the migration script and run it. Then the data are in the new database and the old one can be disabled.

Recently, on the project, which we got from another team, we made a migration from MSSQL to Couchbase. The whole project was reworked under NoSQL for the purpose of simple scalability. Migration from a relational database to NoSQL is not such a simple procedure, as the data structure and the very concept of links change. As a result, a lot of data was lost, including payments from users, gifts and accounts purchased. It turned out that there were errors in migrations only when the system was released. There were about 500,000 active users in the database, so QA couldn’t do a full manual check with the right quality.

How should the migration go? First, we have a new empty database and we need to determine how we will understand that the new database has all the data and connections from the past? This is a non-trivial task, but without solving it, what are we doing? We hope for good luck!

  • data processing and denormalization
    One of the projects I recently described had the following problem in its evolution. The data comes through replication from an external database. These data should be sent via triggers to the DBMS queue, processed, sent to the RabbitMQ queue, and finally saved to denormalized tables. In the course of the project development many nuances appeared. For example, the trigger may not have worked and sent some change to the queue. An error may have occurred while processing the change from the queue. Any mistake gave an incorrect picture for analysts.

Only 3 months after the start of the project were the procedures created that could determine the completeness and correctness of the data transfer. How was it necessary to do it? It was necessary to define and implement these procedures from the very beginning, to type in numbers and metrics to measure the quality of the transfer, and then to deal with the transfer.

  • Crawling
    The history of data crawling in one of the projects is very instructive for me. At first, one team fought for the completeness and accuracy of data crawling for almost a year. Then our team took the project and only after 4 months of development were created analyzers that showed the lags of crawling, its completeness and accuracy.

What was it like to do? First, we should set up monitoring of sources, set up indicators of completeness and quality of data, and then start crawling. At the very least, we should do it in parallel.

General conclusion

There are many stories like this that can be remembered. I am sure that each of you can remember a couple of projects, where the hope for good luck from the first call broke and forced to do well. What do these stories have in common? I think the conclusions about how Acceptance testing should have been reminiscent of. The approach itself is a TDD in the scale of a big task, not a separate module, like an ATDD.

Someone uses Fitness, Selenium with Cucumber and so on to describe the acceptance test before implementing the functionality. Perhaps, in your case you need special projects on collecting and analyzing the necessary data.

Next time, before you start doing something, I suggest you stop and answer the question for yourself and the customer: “How will we understand that we have achieved the necessary results with the necessary quality? How can we measure that we do the system/module/algorithm better than worse? Having answered these questions, we will lay the foundation for effective work.