DevOps and Continuous Integration: a look at BlueMind’s quality approach

The term DevOps is rather recent on the global IT landscape, but it’s already risen to the top among the most effective industry practices. According to the Gartner IT consultancy, in 2015, 25% of 2,000 biggest world corporations were moving towards a DevOps approach. More than a working method, it is above all a cultural change that aims, as its name suggests, to bring development and operations closer together. DevOps has a twofold objective: to speed up update deployment and ensure high quality levels.

Its secret lies in cooperation and sharing: developers and system operations managers work together to provide continuous software delivery. Development cycles are shorter, quality assessed throughout the project and deployments more frequent.
“DevOps is an understood set of practices and cultural values that has been proven to help organizations of all sizes improve their software release cycles, software quality, security, and ability to get rapid feedback on product development.”Puppet Labs, “2017 State of DevOps” report.

BlueMind has chosen to implement a DevOps approach through a process of continuous integration. Let’s have a look at what this entails.

A new paradigm

Traditionally, developers didn’t share their work with the rest of the team and would go off on their own for long periods of time. They’d only integrate code modifications when their work was done. Developers didn’t actually know what the true impact of an update had over the whole application. As a result, issues were only identified down the line and they were difficult to pinpoint, thereby delaying delivery.
As for operations departments, they sometimes had to wait several months for testing and deployment to satisfactorily respond to user needs, which by then had probably changed. Both teams – development and operations — didn’t have the same objectives and therefore didn’t have the same priorities.

On paper, software requirements are clear, set in advance and static. Developers work on the code, and operations teams work on implementing it on the company’s systems. On paper, that is.
In the real world, IT evolves so quickly that it is impossible for project specifications to stay the same throughout. For software publishers, the “project” never ends! Software must be released as quickly as possible, but it must also evolve constantly. Adding new functionalities must as easy as possible and bugs must be identified so that they can be fixed. New features must not disrupt what already exists and take into account all possible configurations and situations. There’s no D-day for product deliveries like in the past.

DevOps practices – including continuous innovation – aim to put an end to the conflict between Dev and Ops – and find a balance between innovation and stability.

DevOps: improved dialogue between teams

The DevOps approach is about setting up an overarching process involving everyone concerned, whatever their initial role. No more isolated and hyper-specialised individuals, development and operations teams work together throughout the entire life cycle of an application, from design and testing to deployment and operation.

Teams share many responsibilities and combine workflows. This limits efficiency losses and saves time.

“At BlueMind, no one is moving forward by themselves. We encourage collaboration and sharing to promote operational efficiency. We have a common objective and a plan – defended by the product or feature owner – we work towards together. The whole team can keep up with projects, ideas and everyone’s stumbling blocks”.

Dominique Eav – Quality Bandmaster at BlueMind

The time spent sharing know-how, ideas and knowledge should largely be made profitable by the efficiency gained in delivery quality, anticipated issues/errors and faster troubleshooting.

Continuous integration

Put simply, integration consists in putting together pieces of code and getting them to work together. This often takes great dedication! Integration often involves multiple projects and environments, each environment involving organising the module concerned differently as well as construction, testing, validation and deployment phases. Continuous integration solves these problems by involving all stakeholders in a project from development to operation.

Continuous integration is the set of practices that allow developers to integrate code changes into a centralised repository regularly, automatically create and run testing sessions, and then deploy build results if all lights are green. This process makes it quicker to detect and fix bugs, helps improve quality and reduce update validation and release time.

Test results history.
For any given test, you can see what’s happening now and in the past. A “failed” on a test that passed before points to a regression

Every time the source code is modified, the result is checked for regressions or defects in the untouched parts of the application. Bugs are identified more quickly and therefore fixed more quickly. Validation and release time are reduced and, obviously, final quality is improved.

“We largely use the principle of pull requests. This mechanism allows developers to share and validate their work whether it’s finished or not. The developer may submit a pull request at any time for review by the people involved. It’s a lot more than mere validation – it’s a genuine discussion about the technical choices for the functionality under review. That way, we make sure that the whole team stays up to date with technical evolutions and choice integrity”

— Dominique Eav Quality Manager at BlueMind.

Dashboard on the build of a (master) branch
This diagram shows the test results for the different build stages and earlier builds

Automation: the cornerstone of continuous integration

Compilation, one-off and operational tests, product validation, performance tests… all are key but time-consuming activities, not to mention susceptible to human error, which is always a possibility. The principle of continuous integration relies on test automation to save time and ensure maximum quality levels.

When a developer writes code, the tool automatically checks that it complies with common rules set for the whole team. This procedure ensures the quality of the newly-written code and its stability over time as it complies with development best practices.

This code analysis tool shows the test coverage on the code. You must seek the highest coverage possible.

BlueMind uses Jenkins to drive its continuous integration chain. This software tests and reports changes made to a large base of code in real time to detect problems quickly.
We use Jenkins to build and test our development branches (features or corrections) whose changes continuously integrate our release branches (3.0, 3.5, 4.0), on all supported Linux distributions (8 for BlueMind 3.5.9). The BlueMind core comprises more than 500 modules and interacts with many AddOns you can find at the Marketplace. These have also been built, tested and deployed through our continuous integration chain and are subject to the same compatibility and quality constraints. More than 3,000 tests are carried out on every build. Such a degree of complexity would be impossible to manage manually! What’s so clever about continuous integration is that tasks automatically run one after the other as we aim for process industrialisation.

“Our approach is consensual and pragmatic. We implement and improve our processes to make our team members’ lives easier, as opposed to complying with an imposed set of project specifications. The resulting automation allows us to be fast and helps potential problems to surface as early as possible. Each brick in our continuous integration chain is a safety belt.”
— Dominique Eav Quality Manager at BlueMind.

This static code analysis tool assesses classes: it measures technical debt, i.e. the time required to fix problems found in some part of the code.

In addition to eliminating tedious tasks, automation makes the deployment process more reliable while increasing delivery frequency. Teams can then concentrate on improving existing features or quality control rather than performing tests manually.

Specific benefits

With continuous integration, development teams save a huge amount of time as each software modification is immediately tested. By compiling code regularly, software modifications that cause trouble are quickly identified. Developers have a quick overview of the regression they’ve caused. Conflicts between the software and the ecosystem it will be operating in are detected early on.
Throughout the development and deployment process, the entire team works very closely towards a common goal: the success of the project as a whole, not just their own part.

When the whole project team is agile, a new functionality or a bug correction can be released quickly – or cancelled as the case may be. A working version of the software is always available and there’s no need to wait for the software to be released to find out whether it works. With BlueMind, we first release new features or software patches internally. This process is called “eating your own dog food“, which means that we are the first ones to use our solution.

On the client side, continuous integration makes deliveries faster with more frequent updates while ensuring the best quality levels possible.

In conclusion

Installing a scheduler like Jenkins isn’t enough to implement continuous integration. One of the Manifesto for Agile Software Development’s statements is that individuals and interactions are valued over processes and tools. A DevOps approach is about involving everyone’s responsibility. The DevOps culture implies adopting a transparent approach based on communication and collaboration between IT/Operations and development teams. It is a great approach for constantly-innovating businesses!

BlueMind was built on these principles from the start to ensure the best quality levels in a complex environment. The agility this gives us enables us to deliver first-class, stable products by industrialising processes between operations and development.

Suivez nous :

Suggestion Box – a new approach to deal with new features

Our job as a software publisher is to listen to our users’ expectations and enhance our product to meet them.

Being open source is what makes BlueMind unique. Yet openness can sometimes be detrimental to its development.

As a result of our product’s success, we’ve had enormous amounts technical and functional feedback and we’ve had to organise ourselves in order to process it.

Business model

Before moving on to the thick of things, we should explain what our job as a software publisher means, our business model and its impact on our organisation.

As software publishers, we focus on software development, product enhancement and technological partnerships. Our target market isn’t country or business-specific. Today, everyone needs email, and BlueMind responds to this need whether you are an SME or a multinational corporation.

To achieve global coverage, we work with a distribution network made up of partners who implement the projects for our clients and/or set up SAAS platforms.

Our network of partners enables us reach a must greater number of potential clients.

This has two consequences:

  • most clients come through our partners, which means we don’t know all of them
  • our number of clients is going to keep growing

As a result, we must set up a series of tools and methods to collect our clients’ feedback and requests in order to process them as best we can.

Organising Development

In the DevOps sense, as shown in this chart by our Normation colleagues, in the process from application development to production implementation, two parts are involved: BUILD and RUN.

The job of a software publisher is to BUILD, i.e. to take business requirements to create “application binaries”. In the Agile sense, business requirements can take different forms (stories, epics, etc.)

BlueMind’s R&D team has been using agile methods since the product’s early days. This has allowed us to develop features quickly, within a specific context. A whole series of automations and tests allow us to deliver new versions with new features quickly.

This article looks at the issue of handling large amounts of business requirements: how can we process them and incorporate them into our R&D appropriately?

Managing business requirements

Why manage them? People ask for things and we do them!

If only things were that simple. We’d need endless resources and all requests would have to be consistent, among themselves and with our vision. In the meantime, we have had to sort through requests and select them to feed a roadmap with a set of features that go into a Backlog.

The backlog contains a number of features which have to be sorted through and validated by a Product Owner. The features at the top of the backlog are processed by our R&D team first.

A simple qualification process

We had set up the following simple and comprehensive feature qualification process (as the young Padawans we were!).

  1. Understand the request
  2. Validate or reject the request
  3. Search for duplicates (who knows whether other users may have made the exact – or not so exact – same request!)
  4. Create a story for the backlog
  5. Place it in the backlog according to its priority status set by the Product Owner
  6. Link it to an Epic, if the story is too big

In theory, this process had 6 steps “only”.

This seems straightforward enough, but it becomes infinitely more complicated when you’re dealing with a large inflow of requests (entered by people who must be clients, but whom you don’t know, much less do you know their vocabulary and their expectations which tend to be too vague). As a result, your backlog becomes, well, backlogged. Looking for duplicates can be time consuming when you’re dealing with a hundred features or so.

In our case, requests come from various sources:

  • end clients
  • our partners
  • internally (developers, sales people, pre-sales people, etc.)
  • the software user community

This adds up to huge amounts of people potentially sending feedback and requests.

people

Most people believe that their requirements are generic, but in most cases they aren’t. Requests aren’t always expressed in the most straightforward manner and everyone thinks their feature is indispensable to their everyday software uses!

Our backlog: on hold

As at January 2015, this was our backlog’s status:

  • 140 features not sorted
  • 470 stories in the backlog

Among these stories:

  • some were important (implementation of a major, structuring feature)
  • others were less important (e.g. evolution of an existing feature)
  • others were small but legitimate (being able to sort by column: feasible, interesting, but high-priority?)
  • others were irrelevant, and possibly clutter (with all due respect to our users “the button would be better in yellow”)

The main issue isn’t so much the flow of feature requests but rather the way they were created and we processed them.

As for all IT projects, we’d set up a bug tracker. In most bug trackers — if not all, as far as we know –, bugs and features are essentially processed in the same way (by default). Typically, the request form is almost identical and a drop-down menu allows users to set the request as “issue”, “bug” or “new feature”.

7-differences

This is where the trouble is: a bug isn’t a feature! They are two distinct processes with different objectives!

Unfortunately, in bug trackers, the difference between a bug and a feature isn’t clear enough.

A feature ends up having the same level of time constraint — and possibly requirement — as a bug, which shouldn’t be the case.

A bug:

  • must have a resolution time-frame and depending on the type of service offered, must follow an SLA
  • must be corrected. It is a fault that must be entered into a correction cycle.
  • a specific workflow must follow its progress (open, pending feedback, corrected, deployed, “won’t fix”, etc.)
  • is critical
  • significantly impacts users, who must be informed of its progress.

A feature:

  • is a request for an enhancement
  • does not have a deadline (contractually)
  • is not critical (contractually)
  • does not require feedback to users
  • can be considered and accepted…  or not

This means that by nature, bugs and features are processed in a dramatically different way.

The Suggestion Boxlogo-sb

This is why we’ve decided to work differently, to reconsider how we handle features completely. First, we won’t call them “new features” or functionalities, from now on we’ll call them suggestions.

As a result of our analysis, we’ve concluded that to optimise how “suggestion requests” are processed, it is important for the requirement to be described as well as possible by the submitters themselves.

This isn’t a technical issue! But how can we get submitters to be more proactive? The vocabulary we use is important, and it is a first step in conditioning users:

  • users will “suggest” rather than “request”!
  • “I want” becomes “I suggest that”, “at some point, I’d like”

This is how the Suggestion Box was created. This dedicated UI is principally designed to browse through existing suggestions.

Credit and discipline

The Suggestion Box must encourage users to ask themselves several questions before they submit their suggestion:

Functional questions:

  • What is the use of my proposed feature and what will it do?
  • What will be its impacts on the BlueMind software as a whole?

Planning questions:

  • Has another user already made this suggestion?
  • Is there a similar existing suggestion?

If a suggestion can be created immediately, it doesn’t encourage users to think about their expectations. Therefore, each requirement turns into a request, with no prior analysis. And it will have to be analysed before the feature is turned into a story (how it is broken down).

This is exactly what used to happen when we allowed the creation of a “new feature” in our bug tracker: “all I have to do is select from a drop-down list and fill in a title”…

We must therefore guide our users through the process, via a suggestion self-service: the Suggestion Box!

Comment

The Suggestion Box was created with the following specifications which are deliberately basic in order to keep the procedure simple:

blueprint-suggestionbox

  • Search

The search is important as the UI’s purpose is to go through existing suggestions. We therefore need a powerful search tool.

  • Suggestion not created if no prior search

To encourage users to search for a similar request, suggestions cannot be created from the home page. A suggestion can be added only if a search doesn’t produce any relevant results.

  • Category

Still in an attempt to get as much information as possible, suggestions must be categorised to help group requests logically and facilitate searches.

  • Votes

Votes are designed to help rank requests. They will help identify the most popular requests.

  • Comments

If a request is similar to an existing suggestion, comments allow users to give another view on the suggestion and provide new arguments rather than create a new one. Adding to existing demands rather than creating a new one is more useful and it allows a suggestion to collect more votes.

  • No status

The point is to allow suggestions to evolve with the community of users and let them become more specific, better-defined and respond to the expectations of as many people as possible. Anyone can add a comment and enhance their vision at any time. BlueMind takes it over at the appropriate time and sets it as “accepted” to draw up the final specifications and carry out the suggestion.

How does BlueMind use the Suggestion Box?

Designing a software RoadMap is a publisher’s job. The Suggestion Box is a component that contributes to the RoadMap among many others. The Suggestion Box is not the absolute basis for the RoadMap, but it will serve to complement the feedback and planned internal projects.

The number of votes for a suggestion tells us how popular a request is. However, BlueMind may decide not to give priority to a suggestion with a large number of votes and prefer a less popular suggestion.

This is because for technical, planning, consistency or strategic reasons, the Product Owner is the one who makes the final decision on a sprint R&D.

Moreover, “internal” tasks, not requested by users (change in API, development library, etc.) may be given priority in order to plan for the possible development of an advanced feature at a later time.

When we decide to upgrade a component (calendar, administration, contacts, etc.), the Suggestion Box is used to find out our users’ expectations and what improvements can be made given a variety of constraints.

Categorising the Suggestion Box allows us to restrict the analysis of suggestions to the module we have chosen to upgrade.

Bottom line

Two years after the Suggestion Box was launched, feedback is extremely positive.

Today our backlog includes 78 stories broken down into 22 EPIC while previously, we were struggling with 470 stories in backlog and another 170 non-analysed features.

The Suggestion Box includes 224 suggestions, 662 votes have been made  on 134 suggestions. 50 suggestions have been carried out, some with many votes (up to 46), some with fewer votes but that were considered high-priority.

Additional Information

You can find BlueMind’s Suggestion Box at https://community.bluemind.net/suggestions/

Our development is OpenSource and you can find everything you need to know about this project at http://www.suggestion-box.io

The Suggestion Box is the result of a lot of in-house discussion and reflection, in particular with Dominique Eav, our quality manager and developer of the Suggestion Box.

And, as we keep updating the Suggestion Box, you’re welcome to create a suggestion for the Suggestion Box!

Enregistrer

Enregistrer

Suivez nous :

Tutorial: writing a BlueMind add-on

So here you are with your shiny BlueMind setup. You’ve heard about our grand architecture plans, extension points, p2 platform, REST API, but you find it hard to get started. Rejoice! This article has been written just for you.

You need to know some java, and be more or less familiar with maven.

Your goal: a dummy scheduled job

Let’s say your BlueMind add-on will be a scheduled job. You can view the BlueMind scheduler as an internal CRON that will execute jobs when planned, which is pretty handy to assist you in administrating your server.

This scheduled job will log some dummy stats – basically what we’ll do is just a placeholder to demonstrate how to glue the REST API bricks together. Doing something more meaningful will be left as an exercise to the reader 🙂

Your scheduled job will list all mobile devices for all users for all domains of your BlueMind server. Just because you can. We’ll call it MobileDevicesListingJob.

A word of caution

Don’t forget that when you’re using the REST API with a BlueMind server, you’re dealing with real users data and it’s pretty easy to make mistakes, since everything you can do in BlueMind can be done using the REST API. There won’t be a confirmation screen like we do in the Administration Console: when in doubt, just don’t. Or better: do your API tests on a sandbox server. We won’t be modifying data in this tutorial, so you should be safe, but you’ve been warned.

Bootstrapping a maven project

If you take a peep into BlueMind internals, it’s OSGi bundles all over the place. Since we want to build a BlueMind add-on, we depend on BlueMind target platform.

This target platform is published as a p2 repository, and you can find it at http://pkg.blue-mind.net/p2/latest/

As the URL implies, this is the latest and greatest BlueMind target platform. It will get updated every time we publish a new BlueMind release – and hopefully upgrading won’t break your work.

We’ll be using Tycho, which makes it easy to build your bundles with maven. Here is the simplest project_structure. Download it and extract it somewhere.

This project is restricted to the bare minimum. I won’t delve into the details of the configuration files since they are not BlueMind specific, but I want to emphasize two points:

  • pom.xml declares where to find the BlueMind target platform
  • plugin.xml declares what extension point we’ll be using, namely scheduledjob_provider, along with the name of our soon-to-be-written java class: org.example.mobiledeviceslistingjob.MobileDevicesListingJob

The rest is just boilerplate configuration.

Alternatively, you could also use our maven archetype to bootstrap your project. Or clone bluemind-samples to find an existing add-on to adapt. There are more than one way to skin a cat. Or a duck. Well…

In the project directory, you can run:

mvn install

Et voilà! You’ve built your project, but it won’t work yet: there’s no code.

Eclipse to the rescue

You may just go ahead and write the java code using your favorite editor, but here is how you’ll do with Eclipse:

  1. Import your maven project into your workspace
  2. Tell eclipse about the BlueMind target platform: go to Preferences, search for “Target Platform”, and add a new target platform definition. You need to start with an empty target definition, pick up some name (for instance “BM target platform”), then add the URL above as a new Software Site location:

screenshot-from-2017-03-29-17-43-39

Select all (the “Uncategorized” checkbox in the previous screenshot), click on Finish and you’re done. Careful, you need to select the target platform we’ve just defined before you close the Preferences window.

Setting up the dependencies

We need to declare what parts of the BlueMind target platform we’ll actually use in the MANIFEST.MF. Our REST API online documentation for will help you to cherry-pick the parts you’ll need. If you’ve installed the optional bm-docs package and given the Api docs permission to your user, this documentation should be available right in the BlueMind UI:

screenshot-from-2017-03-30-09-36-52

Icing on the cake: this inline documentation is interactive, so you can execute calls using the javascript client. Beware, the previous caution notes also apply! (the harm you’ll be able to do depends on your logged-in user’s rights)

The BlueMind source code is also be a good place to find inspiration.

Here are the dependencies we’ll need, just add these lines at the end of the MANIFEST.MF:

Require-Bundle: net.bluemind.domain.api,
 net.bluemind.user.api,
 net.bluemind.device.api,
 net.bluemind.scheduledjob.scheduler,
 net.bluemind.core.rest,
 net.bluemind.slf4j
  • net.bluemind.*.api : the REST apis we’ll need to explore domains, users and mobile devices
  • net.bluemind.scheduledjob.scheduler: to make use of the extension point
  • net.bluemind.core.rest: to setup authentication to the REST API
  • net.bluemind.slf4j: logging classes – since we won’t do much more than log some information at the end of the day

Have fun with the REST API

So go ahead and create the class we’ve declared in plugin.xml: org.example.mobiledeviceslistingjob.MobileDevicesListingJob. It has to implement IScheduledJob for scheduledjob_provider to be able to plug it in. Here is the complete code. I will only include here the actual business logic:

// logger will write in bm core logs (/var/log/bm/core.log)
logger.info("Executing MobileDevicesListingJob");
// sched will write in the execution report, that you can send by mail in the Job setup UI
sched.info(slot, "en", "Collecting mobile devices data for all users...\n");
// write header row for the data to come
sched.info(slot, "en", "device; isWipe; lastSync; user; domain");
// initialize client for domain service
IDomains domainService = ServerSideServiceProvider.getProvider(SecurityContext.SYSTEM)
    .instance(IDomains.class);
// loop on all domains
domainService.all().stream().forEach(domain -> {
  // initialize client for user service
  IUser userService = ServerSideServiceProvider.getProvider(SecurityContext.SYSTEM)
      .instance(IUser.class, domain.uid);
  // loop on all users
  userService.allUids().stream().forEach(userUID -> {
    // grab full details for user
    ItemValue<User> user = userService.getComplete(userUID);				
    // initialize device service for each user
    try {
      IDevice deviceService = ServerSideServiceProvider.getProvider(SecurityContext.SYSTEM)
          .instance(IDevice.class, userUID);
      // loop on all devices
      deviceService.list().values.stream().forEach(device -> {
        // collect info for this device
        List<String> deviceInfo = new ArrayList<>();
        deviceInfo.add(device.displayName);
        deviceInfo.add(Boolean.toString(device.value.isWipe));
        deviceInfo.add(device.value.lastSync.toString());
        deviceInfo.add(user.displayName);
        deviceInfo.add(domain.displayName);
        // write a line in the report					
        sched.info(slot, "en", String.join("; ", deviceInfo));
      });;
    } catch (ServerFault exception){
      // Skipping this user since she doesn't have a "device" container
    }
  });
});

 

Deploy to your BlueMind server

You can compile this code with

mvn clean install

Then drop the resulting jar (target/org.example.mobiledeviceslistingjob-1.0.0-SNAPSHOT.jar) in the folder /usr/share/bm-core/extensions of your BlueMind server. Then you can restart bm-core

/etc/init.d/bm-core restart

And your job should appear among the Scheduled Jobs in the Administration console. You can then execute it, schedule it, send the report to your best friend, it’s all yours.

Caveat: if you need to recompile/redeploy your extension, you may need to delete bm-core’s cache to be sure the fresh jar is picked up:

rm -Rf /var/lib/bm-core

Talking REST from the outside world

You’ve noticed we’ve used a ServerSideServiceProvider to initialize the REST API services, but you can of course talk REST from the outside world, for instance by using our python client. You need a BlueMind API key to do so.

Share your work!

Don’t bother sharing your MobileDevicesListingJob, but once you’ve done something useful you may want to share it on BlueMind Marketplace. We’ll be looking forward to your contributions!

Suivez nous :

BlueMind turns Thunderbird into a true collaborative client!

bluemind-thunderbirdA frequent obstacle to the adoption of an Open Source mail server is the predominance of Outlook as a thick client because Outlook always works differently – and therefore more or less smoothly – with severs other than Exchange… until BlueMind’s soon-to-come innovation which will bring native Outlook support!
This is a matter of user habit, but this is also due to a virtually inexistent offering of good open source, multi-platform email clients (for email and collaboration).

The most popular open source email client is Mozilla’s Thunderbird which runs on Windows, Mac, Linux and others.

But although Thunderbird is a good email client, it is a poor collaborative client. Its calendar feature is provided by an extension – Lightning – which has never been entirely satisfactory for enterprise use (in terms of ergonomics, performance, etc.): users fail to understand having to use two different interfaces with different ergonomics and features depending on how they access it (web or thick client).

BlueMind offers comprehensive, user-driven Thunderbird management, which precisely addresses these issues.

Discover Thunderbird as BlueMind’s natural thick client!

Continue reading “BlueMind turns Thunderbird into a true collaborative client!”

Suivez nous :