DevOps and Toolsets – Teaching Performance Engineering

Article
02/28/2017

I have been working with a University over the past few weeks helping them implement a new course on Performance Engineering which encompass a lot of DevOps practices, think the teaching and awareness of DevOps is a excellent, within Industry there are real consequences if an organisation is unable to deliver apps or services quickly or builds a completely wrong implementation from the start. So I was very excited to be involved in the course design and wanted to share some of the outcomes:

Some interesting Research stats

Over 40% of software implementations end up getting reworked because they don’t meet the users’ original requirements or these requirements have been misunderstood

The average cost of one hour downtime of a customer-facing app is calculated at 100.000 dollars per hour – and this does not take into account the damage to reputation, which can be even greater.

Fixing such production issues takes on average 200 minutes per incident

Agile Teams

But stats also indicate that over three quarters of development teams have adopted Agile methodologies today, enabling them to develop faster. But this can present a number of problems for the team, a simple lack of collaboration between dev and ops can have substantial impact on a company’s bottom line and success. DevOps is not a new concept – it’s been around for a couple of years and it grew out of the Agile movement. A converged DevOps lifecycle gives you the ability to execute on ideas quickly and iterate on feedback rapidly, with minimum friction while at the same time maintaining the highest level of quality

Key goals of DevOps

1. Shorten your development cycles and deliver better apps, more frequently

2. Achieve faster development cycles with state-of-the art DevOps tools and an agile, end-to-end, tools-to-platforms solution

3. Streamline application delivery and boost the productivity and quality of any mobile project with a full range of integrated, mobile-specific QA tools, including test management, exploratory testing, performance testing, automated UI testing on thousands of real devices, and beta testing

4. Close the loop between development and production, fix bugs faster and evolve your application over time with insights and learnings captured directly in production.

One of the key things around DevOps in the understanding of tools, services and technologies

DevOps and Tooling

There are two key views on this one from a complete Open Source Tooling perspective where organisations have the following technology and services in operation

Or those who are working around the Microsoft technology stack

Both are perfect solution to organisations and off course a hybrid approach of technologies is also suitable.

Course Structure

The academic or TA – Becoming/acting as Customer for testing of users services

The academic has a dedicated Azure subscription and implements accounts for students within the course.

The academic has a dedicated single point for assessment/validation – i.e. checking up-time which student has the most available service

The academic has central service which can drop loads test to the students services as and when required, simulating real life loading and scenarios.

Testing of the Student provided services be Working hours i.e 9am – 5pm to allow cost/scaling/predicative modelling

Azure DNS provisioned in the academics Subscription – This subscription will need delegated rights to students record to ensure student IP address are correct for continuous testing/pinging as each machine provisioned will get a new IP address.

Potential Test Scenarios

Availability

Geographic Response Time

Geographic Load

CDN static file web test.

SLA failure - Failed X concurrent pings – Fatal Error – Under X pings – Transient Error

Testing of End Points.

Access Web Resources Known and Not Known

Database provision of user – New Accounts i.e Database back end requirements and authentication

Semester 1

Start with VM Dev/Test to Production Environment

Implement a basic Web Service/App

Tier Application Learning – i.e Start with single tier and work up through the phases

Implement monitoring/benchmarking/Bugs in potential Apps which can be discovered through logging

Identifying issues through monitoring

Financial Modelling and Cost predictions i.e Autoscaling cost for month 1 $300 for Month 2 $200 Month 3 $100

Development of SLA – Which can be compared with availability test

AB testing – continuous service ping to replicate Live Service

Alerting and Monitoring

Semester 2

Scale out this services to multiple VMs Scale up/out MutiTier – i.e Benefits/Risks/Costs

Implement Load Balancer

CDN, REDIS

Implement Multi Tier app with State Less app

Persistent session

Availability i.e simulated work loads i.e scale in/out/up/down

How does this effect your metric Availability/Budget/Allocations

Financial Modelling and Cost predictions

Development of SLA – Which can be compared with Ping Tests

AB testing – continuous service ping to replicate Live Service

Alerting and Monitoring

Semester 3

Scale out to SaaS or Containers..

SLA/Quality of Services

Metrics and Informed Decisions from tooling

How does this effect metrics budgets/resource

Comparison of Services offerings

Financial Modelling and Cost predictions – Scale Sets/Maximising Budget

Development of SLA – Which can be compared with availability tests

AB testing – continuous service ping to replicate Live Service

Alerting and Monitoring

Evaluation and assessment

From each of the semester evaluation and activities, there should a quantified outcome of performance from both a implementation and operations perspective and students can easily be stack ranked in terms of effectiveness of meeting the business requirements and brief.

DevOps from a organisation perspective

Every DevOps discussion in any organization has to focus on 3 elements: people/culture, process, and tools so for anyone who has used or studied ITIL this can be related simply into ITIL processes.

1. People/culture is about bringing people together in a collaborative fashion that enables multi-discipline teams to work together more easily, share common goals, and focus on identifying areas for improvement

2. Process is defined as finding ways to increase efficiency, streamline feedback and eliminate waste All software development processes should be focused on delivering business value faster - focusing on the flow of value

3. Tools are arguably the least important part of this discussion – they support processes and facilitate continuous feedback and experimentation Tools are a way of executing the strategy and vision that an organization has formulated with regards to their journey to Dev.

So What is the DevOps Lifecycle

Every project starts with an idea - and a plan how to turn this idea into reality

Once the project is under way, work needs to be managed and progress should be tracked continuously

Stage 1.

The goal of this stage is to build high-quality apps or service the users expect the best experience everywhere and do not tolerate bugs

For mobile apps each platform has different presentation styles, interaction styles and software stacks

Devices have different screen sizes, input modes and hardware capabilities

New devices and OS versions are introduced multiple times per year

Network connectivity and power levels can fluctuate widely in typical usage scenarios

New consumer applications regularly extend and revise the standards and set the bar higher for good mobile applications

Many businesses have already started their journey to mobile with consumer apps some time ago, so they are familiar with some of the biggest challenges

Stage 2.

Once a sprint starts, developers turn great ideas into reality by writing code and building functionality

Before checking in code to source control, unit tests are created and run to ensure that quality bars are met

Ideally, every check-in by a developer triggers a build and automated build verification tests ensure that the end result is a working iteration of the product

Building high-quality apps is hard – but users expect the best experience everywhere and do not tolerate bugs

Each platform has different presentation styles, interaction styles and software stacks