The improvement of software factories is one of the current OCTO’s R&D topics. This article presents our vision regarding their future. Before diving into new features and perspectives, let us first describe the concepts and limits of current software factories.
A software factory is made of several tools for both development and automation of the overall software building process, from development steps to production delivery. A software factory also contains all the tools required for software development, thus including documentation, wiki and source code manager. Software factory is important in development industrialization : it is actually the keystone of a continuous integration. Indeed, state-of-the-art software development follows a process based on the software factory. It provides a better control of development quality and progress through indicators.
The following diagram provides an overview of a software factory :
The represented workflow is quite complete and answers the needs of many projects. The choice to actually implement each feature depends on the project maturity.
The “defect cost increase” states that the later an anomaly is detected, the more expensive it is to fix it, with increase magnitude ranging from 1 to 1000. So, early bugs detection is a real gain, which can be achieved through continuous integration with a software factory. A software factory is a powerful tool for development and allows the automation of the following steps :
Checkout of reference code. Moreover, it reinforces the use of a professional source code manager.
Source code compilation
Run of automated tests
Measure of test code coverage and code quality
Respect of development standards
Check the good use of design patterns
Compliance with architecture rules
A software factory also induces additional benefits :
The most advanced software factories provide automatic deployment and driving of development, through features such as :
Although a software factory is first and foremost used by development teams, its influence zone extends further. Indeed, 4 main profiles are identified :
The industrialization of a project raises many challenges at different levels. The first and most important one is to automate as many tasks as possible. It is one of the main goals of the DevOps stream because it minimizes the risk of human errors. In the long run, time is better spend on maximum automation than on correcting oversights and typos... that people generate from time to time! Regarding to the quality of development, its goal is to improve the consistency and to establish some rules and standards that can be quickly applied. At Operations level, delivery success is guaranteed by optimizing the automation of the delivery process. Everyone has been faced with stories similar to “oh ! I forgot to change the properties in the 42rd properties file of prod-18 environment!”.
According to this, identified priorities concern both code lifecycle and management tasks such as driving and reporting of activities. Finally, a software factory is also very important to avoid technical and functional regressions but also performance regressions. The later requires the ability to be able to test and to measure the performance of the software being developed.
All these improvements lead to a better time to market (TTM).
For a long time, IT teams maintained bigger and bigger projects, which imply higher maintenance costs, longer time passed into the software factory's workflow and higher bug probability. Time to market could be improved with a better build time (compilation, tests, quality…). However, unless a complete code refactoring to allow parallel test runs with tools such as Maven 3, build time optimization limits are quickly attained. We have identified several other limits on the current model :
Continuous deployment (the real one) : many software factories are already used to deploy continuously, but it’s often a custom deployment script that is called by the software factory.
Multi-platforms management : iOS, Android, .NET, Java… impossible to build all of them in the same environment.
Cloud deployment of all parts of the complete software factory is far from easy and reproducible, even if cloud software factories exist (see this article .
Generating a complete software factory per project typology (Java or iOS…) can be imagined.
Those software factories need to be adaptable and multi-projects : software factory mutualization is a must. One software factory per project is too expensive to maintain and therefore to be considered as a viable option.
Monitoring : for example, following the time to resolve in real time.
Unbreakable build : it is already possible in current software factories, it is a requirement for continuous deployment.
Build distribution on several nodes : possible on some platforms, setup complexity is high. Nevertheless, it should be a good way to speed up build times. (Time to market focus)
DevOps is a set of practices intending to improve collaboration between development and operations teams (See : [this article](http //blog.octo.com/devops-ou-le-lean-applique-aux-activites-it-du-developpement-a-la-production/)) and tends toward a responsibility of every project actor. The DevOps stream offers tools dedicated to these tasks, especially for implementing and measuring automation. This brings the idea of a very promising software factory 2.0 in which the DevOps tools are mixed with the actual software factory in order to answer to the following issues :
Today, there are still many manual steps that could be automated through DevOps tools (server deployment, configuration management...). In the same way, validating the required but manual steps (UAT, deployment decision...) could be facilitated through the DevOps tools.
Widely used by DevOps, CMDB tools (Configuration Management DataBase) can centralize the environment configuration, but also integrate it into the deployment automation (we're talking about deployment to run the various tests but also for production!)
These tools are not just a fad because the server management tools such as Chef or Puppets are not only highly productive but also safer. Indeed, they prevent the classic "oversights" when setting up a new machine. They allow the automation of the setup of a server with the packages and the configuration necessary for the wanted services, so why not a setup for a Software Factory or a part of it!
Nothing new for testing purpose, but the integration of modern instrumentation tools for testing and separation of these various tests is an important factor in the new software factory.
Not always in place in current software factories, this dimension is still very important so it must be part of the new software factory.
Making performance tests is good, but running them regularly to be able to detect a decrease of performance from day to day is better. Even though, it is often very expensive and time consuming. However, the tools presented above should facilitate their integration into the build.
Build time is too long? On demand parallel execution or distribution on multiple nodes can be a solution. The idea is to reuse the patterns found in other applications (clustering, load distribution...) to build them.
Once all steps are cleared, let’s deploy to production. The benefits of continuous deployment have been proven many times.
With all the features that will be managed by the software factory, traceability is a must-have in order to understand why things have been done but also to be able to set up an archiving system. Another important element is the ability to go back (rollback) if the deployment goes wrong.
Indeed, Multi-technology IS is a real management issue : Java, NET, iOS, Windows Phone... each compiler needs its own platform but this goes against the wish of having a shared software factory...
To be continued into the next episode... stay tuned!