Today, businesses increasingly rely on software. There is a strong need to deliver software on-time and on-budget. It is no longer acceptable to postpone customer feature needs or launch a product after the right time has already passed. Therefore, we cannot rely on traditional methods of development/test (dev/test) and IT operations (ops) that operate in siloed environments with work that is “thrown over a wall,” lacking automation, and without standards.
DevOps
DevOps answers these needs for agile software development by emphasizing automation, monitoring, and communication and collaboration between developers and operators. It lets developers spend more time developing applications and gives testers more time ensuring code quality. Operations can focus on “keeping the lights on” and troubleshooting high priority issues instead of attending to mundane tasks. DevOps emphasizes configuration in code and automation in provisioning.
With DevOps, apps are moved from dev-test to production 24x7 through a process called Continuous Integration/Continuous Deployment (CI/CD). Continuous Integration (CI) emphasizes catching issues early and often, versus the previously held notion that issues should not even arise in production environments. Continuous Deployment (CD) emphasizes getting a product into production early. In a CD environment, websites and code updates are made multiple times daily, sometimes in the order of minutes. Practitioners of CD include Facebook, LinkedIn, Etsy, Amazon, and others. For further details on how CI/CD is benefitted in DevOps, specifically at NetApp, scroll down to CI/CD Using OpenStack@NetApp as a Case Study.
The Challenge with DevOps
A fast-paced DevOps environment emphasizes agility through continuous integration and deployment. This agility can be achieved through programmatic access to resources, storage efficiency, seamless scaling, non-disruptive operations, data protection, and other enablers. NetApp’s Clustered Data ONTAP storage systems with OpenStack Manila provide feature-rich file-share service storage that is ideal for DevOps.
OpenStack Manila
OpenStack Manila is the file-share provisioning and management service from OpenStack. Users can use a self-service API to provision and manage NFS and CIFS file-shares on NetApp CDOT storage systems (aka FAS storage systems).
NetApp Clustered Data ONTAP (CDOT)
NetApp Clustered Data ONTAP (CDOT) provides an enterprise-class backend for Manila through the use of Storage Virtual Machines (SVMs). Data ONTAP FlexVol volumes are created in SVMs, and FlexVol volumes and Manila shares are analogous.
CI/CD Using OpenStack@NetApp as a Case Study
The OpenStack team at NetApp operates in a DevOps model and is a great example of Continuous Integration (CI). Our code changes are tested multiple-times daily using a Jenkins-managed test suite that executes tests on CDOT and E-Series storage systems. When a new code change is submitted to our code-review system, the system analyzes the code changes and chooses the appropriate test cases to run against this change. Based on the results of this test, the dev who made the change is notified whether their patch is ready to be promoted to the upstream OpenStack Github repository. If the CI system votes “yes,” only then is the change is ready to be promoted. The process for code submissions shown below.
The OpenStack team at NetApp also implements Continuous Deployment (CD). Code is reviewed using automated tests, tested for quality, and peer reviewed by other devs and continuously pushed “upstream” to the OpenStack github repository. For example, the upstream repo for Manila is located here on GitHub.
Advantages of using OpenStack Manila+Clustered Data ONTAP for DevOps
In order to truly see the value of Manila with DevOps, let us walkthrough different possible scenarios in DevOps team, and consider some examples at NetApp.
Programmatic Access to/Provisioning of Resources Using NetApp Engineering as a Case Study
DevOps teams can interact with OpenStack using its REST API and CLI that allow configuration using simple REST calls and shell commands respectively. Take the following scenario: your OpenStack DevOps team has a requirement to programmatically interact with Manila shares to create and delete snapshots, and to create new shares from these snapshots. You can do this programmatically with Manila.
A great example of programmatic provisioning of resources is the DevOps department at NetApp Engineering. This department provides compute and storage resources to CDOT developers. Code is pulled from a popular OpenStack distribution and used to deploy OpenStack controllers and compute instances. When a new region needs to be provisioned or an existing region has to be expanded with instances and storage, a test environment is first setup and provisioned using Puppet. Once configuration is verified by a DevOps engineer, automated provisioning of OpenStack compute and controllers (also using Puppet) is kicked-off. At NetApp Engineering, all hardware is standardized by number, configuration, and type within and across regions to simplify deployment and debugging.
Storage Efficiency using NetApp Snapshots and FlexClones
Let us walk through a common scenario in every DevOps shop. A new dev, Alex, joins the team and he needs his dev/test environment to be provisioned. Alex can login to the team’s CDOT storage system (aka NetApp filer) and create a clone (NetApp FlexClone) of a fully functional dev/test environment. NetApp FlexClones are extremely space-and-time efficient, i.e. they consume only a small-fraction of the space required for the entire dev/test environment and can save significant amount of time in a fast-moving DevOps environment. Further, by placing the dev/test environment in a Manila share (or importing it to Manila), Alex and his team can take advantage of programmatic calls such as taking snapshots and creating and deleting clones from those snapshots through automation.
In another situation, consider a customer reporting a bug with a particular piece of code to the product dev. In order to reproduce the issue, Alex can quickly spin up a dev/test environment using Manila snapshots. This feature can significantly boost productivity and responsiveness in any DevOps shop.
Seamless Scaling for Local Code Repositories
A Manila share can also be used for for code base in local code repositories, such as GIT, Perforce, etc. When devs and testers have to pull code from the repository, they can take advantage of faster download speeds available in your corporate LAN instead of using repositories on the Internet. As your repository grows, you can increase the size of your Manila share seamlessly. Conversely, you can also decrease the size of your Manila share repository via the CLI or REST API.
Continuous Operations
Administrators can non-disruptively move FlexVol volumes (flexible volumes) between different aggregates (sets of disks). This can be extremely useful in a variety of situations. Consider a database workload in a DevOps department leveraging an NFS share. Assume that this NFS share is running on spinning disks. When your IT admins notice high levels of IO utilization, they can move that database workload to SSD disks without impacting the business.
Data Protection
Manila will soon allow the use of share replicas. Replicas allow copies of code-base or dev/test environment to be placed in multiple availability zones and replicated. You can use replicas to provide high-availability of your resources. When combined with NetApp Snapshots, this can provide a very effective high availability and disaster recovery (HA/DR) and backup strategy.
Note: Until the Manila share replicas feature is publicly available in the Mitaka release (mid 2016), please use NetApp SnapMirror technology. Snapmirror can configured directly on a NetApp FlexVol volume, using either the CLI or OnCommand System Manager.
Other Advantages
Unified Architecture
Using CDOT for your file-share service provides value beyond traditional ways of doing Unix/Unix-like and Windows file-sharing. You can use the ONTAP platform with Manila to provide both of these capabilities within your organization. You don’t need to build and manage separate Linux and Windows file share servers and associated storage. In a DevOps environment, this means that you can use Manila for both Windows and Linux development and testing (dev/test).
Secure Multi-Tenancy
You can place your Manila shares on different disk-pools (or even storage virtual machines) to provide secure multi-tenancy. Consider different organizations in your company that want their data separated from each other for compliance, security, or other reasons. Operations (Ops) can configure this on NetApp CDOT by using separate disk pools (aggregates) for these Manila shares. They can enhance security further by having these shares on separate storage virtual machines (SVMs) to provide separate logical interfaces (LIFs).
Takeaways
There are numerous reasons to use NetApp Clustered Data ONTAP (CDOT) with OpenStack Manila for DevOps, including programmatic access to/provisioning of resources, storage efficiency using NetApp Snapshots and FlexClones, seamless scaling for local code repositories, continuous operations, data protection, and more. OpenStack@NetApp and NetApp Engineering are examples of two teams that use NetApp integration in a DevOps environment. As new features get added to Manila in the regular 6-month cadence that is the norm for OpenStack, the reasons for using Manila with CDOT for DevOps will become even more compelling.