If you’re at DockerCon Europe next week, in beautiful Barcelona, Spain, make sure you stop by the NetApp booth to talk with us about how to use NetApp storage for persistent data with containerized applications, we love to talk about how we can protect your application data. One of the exciting integrations we are demoing uses NetApp’s Snap Creator Framework to manage quiescing, snapshotting, and restoring persistent data for applications. We do this using some proof-of-concept code which tracks the pertinent information between the containers, the hosts, and the storage system, which is explained below. This is a research project to more closely understand data protection with containerized applications. If you’re interested in using this in a production setting, please stop by our booth (#29) or send us an email.
Why is this important in the age of containers? Because, regardless of how the application is deployed, protecting data is always critically important. Giving the administrator/developer the ability to manage their own snapshots and recovery operations removes bottlenecks at the infrastructure layer and encourages more dynamic operations. How many applications want to wait on the storage admin to do a restore for them? For applications which are using a continuous integration or continuous deployment model, is it feasible to wait on anyone? Enabling self-service for the administrator and developer teams means they can handle issues themselves, and that means they can move as quickly as needed to keep the application operating.
With the agent we build and track an inventory of the NetApp volumes and the container(s) which are using them. This agent then automatically registers the container with Snap Creator as it’s created. This allows the container to be created anywhere, but still have its backup and recovery controlled from a central location, the Snap Creator server. Let’s look at this in more detail.
Building an Inventory
When a new container is instantiated, the first step is to ensure we know which storage resources the container is utilizing. The agent does this by collecting three pieces of information:
- The host’s storage devices
- The NetApp storage which backs those host storage devices
- What local devices the container is using
The first step is to determine which devices are coming from a NetApp. This is done by getting all of the storage devices on the Docker host and finding out which ones are non-local. The simplest method of doing this is with the mount command. Once the map of local device to NetApp object has been determined we want to find out which devices the containers are using.
By querying the container configuration using docker inspect and searching the result for host storage devices which have been passed to the container we can quickly and easily determine which containers have NetApp storage in use and which FlexVol or LUN it is.
Registering Containerized Applications with Snap Creator
At this point we now have a map of container storage to NetApp storage. Using the Snap Creator API, we simply register the application in the container with the Snap Creator Server. After this step, the application appears in the management console and is ready to be managed.
There are two additional things we need to know in order to complete our Snap Creator auto-registration. The first is that we need to subscribe, or poll, the Docker events stream. This will tell us when a container is created, which we can then use to poll Docker for additional information, such as the volumes in use.
The last thing we need to know is which containers have applications we care about and what Snap Creator plugin to use. This can be managed with a simple configuration file which maps the name of the container with the name of the Snap Creator plugin.
There are a couple of things missing here, like if we need access to username/password combinations, but there’s nothing that prevents us from further querying the Docker daemon for container information or integrating with a secrets system. This expands the possibilities of the Snap Creator integration to include nearly any application while still maintaining some degree of security for the credentials used. Remember that Snap Creator is a framework, which means it can be used in a nearly infinite number of ways to integrate with your application and infrastructure.
Protecting Your Applications Your Way
As you can see, the NetApp data protection portfolio enables you to protect your application data on your terms, even when those applications are being deployed using a new method. As more applications begin to adopt the microservices architecture the overall implementation will become more complex, but that doesn’t mean the ability to protect and recover data isn’t critically important. Snap Creator’s ability to coordinate snapshots across many different volumes and applications means that all of those components can be coordinated and quiesced with a minimal amount of effort: it only needs a simple plugin to understand the application component, and for many of the most popular databases and other software packages plugins already exist.
Containers are exploding in popularity. They simplify many operations which previously caused friction between development and operations teams, making them a valuable endeavor for nearly any organization, big or small. Further, they expedite the deployment process, making the transition from development to production nearly seamless. NetApp is the storage platform of choice for providing persistent storage to your containerized application, with robust protection and unmatched data management features, stop by booth 29 at DockerCon Europe, or reach out to us at opensource@netapp.com, and we will be happy to discuss the best way to leverage NetApp and container technology together.