Performance Testing; Network vs Storage

 I’m sure we’ll all been there; you are working on a ‘performance issue’ and another technician tells you, “I don’t know what the problem is, ping response time is fast and ping didn’t report any dropped packets.”

I’ve written and spoke about this before, but want to briefly cover my concerns surrounding ping. I will start with a simple statement that I am not ‘anti-ping’ and actually use it quite often. My concern is that I see it used in the wrong scenarios and its operation is widely misunderstood.

Things you should know about Microsoft’s ping command include that they are sent at a 1 second interval, uses ICMP as its protocol and neither option are configurable. Other default settings you can change are default payload size (-l), TTL (-i), disabling resolve to address (-a), setting do not fragment (-f) and forcing to just use IPv4 (-4).

I personally prefer to use hrping (https://www.cfos.de/en/ping/ping.htm) as an alternative to Microsoft’s ping since it has many more options and still a command line tool, even though hrping also provides a GUI interface as well. This utility can send UDP packets, send a ping size ‘sweep’ of various sizes, send ICMP timestamp requests, interval between packets as well as the same options as the Microsoft ping command.

In summary, I use ping to validate or verify if the network and hosts are responding and if there is general packet loss. If the ping test results in no packet loss and response time is acceptable, I test the client or server performance.

If you want to test network performance, nothing beats iperf3 (https://iperf.fr/) but if you want to test the file system, copying a file is the easiest test. I know, this isn’t the same as sending hundreds of transactions, etc. but it’s a great start. In this video I show you how I used a simple Powershell command (Measure-Command) to time how long it took to copy files to/from my server.

In this video, I show you how I used iperf and a power shell command to copy a file (both read/write) on two different shares to identify if the issue is file system or network related.

Microsoft Command line and Wireshark utilities

 I get quite a few emails expressing frustration and confusion when people try some of the Windows Wireshark command line utilities like tshark, dumpcap and editcap to mention a few.

The main issue is that many times the Wireshark application folder is not in your path. There are many ways to get around it like

- Change drive and directory to the Wireshark application folder from within the command prompt

- The use full application path for the wireshark utility you are trying to run (ie c:\program files\Wireshark\tshark -D)

- Add the Wireshark application fold to your path – which is the one I will cover in the video

I prefer adding Wireshark to the path so I can type the command from any folder, anytime without having to worry about which folder I’m in as well as typing the fill path with the command.

A Quick Guide to Kubernetes Persistent Storage (Gilad David Maayan )

 

What Is Persistent Storage?

Persistent storage, or nonvolatile storage, involves using data storage devices that can retain data after the device is powered off. The Kubernetes architecture initially offered only ephemeral storage volumes that were bound to the lifecycle of containers.

This type of storage works for stateful applications but does not allow you to retain data beyond the scope of the container. Later versions of Kubernetes storage introduced persistent storage volumes that enable you to build stateful applications.

Persistent Storage in Kubernetes

Containers are ephemeral by nature, meaning that they are started for a specific purpose and shut down when complete. On their own, containers don’t hold state data, and a new container instance has no “memory” of previous instances. A container does have storage, but it is known as “ephemeral storage” which is wiped out as soon as the container shuts down.

As developers adopt containers for additional use cases, there is a need to manage persistent storage as part of containerized applications. For example, a developer might want to run a database in a container, and have the data stored in a volume that persists even after the container shuts down.

Kubernetes, the popular container orchestrator, provides numerous management capabilities for groups of containers, called clusters. One of those capabilities is the ability to manage persistent storage. Kubernetes persistent storage allows administrators to maintain several types of persistent and non-persistent data in a Kubernetes cluster. Storage resources can then be used dynamically by multiple applications running on the cluster.

Kubernetes provides two primary mechanisms to help manage persistent storage:

• Persistent Volume (PV)—PV is a storage element, either defined manually or created dynamically based on a storage class. It has an independent lifecycle, which is not determined by the lifecycle of Kubernetes pods. A pod can mount a PV, but when the pod shuts down, the PV remains and its data can still be accessed. Each PV can have custom properties such as type of disk, storage tier, or performance.

• Persistent Volume Claim (PVC)—PVC is a storage request by a Kubernetes user. Any application running on a container can request storage, specifying the size and other characteristics of the storage it needs, based on the custom parameters (for example, SSD storage). The Kubernetes cluster can then provision a PV based on available storage resources.

Lifecycle of PV and PVC

Kubernetes offers the following provisioning options for persistent storage:

• Static provisioning—cluster administrators can create PVs that offer storage resources. These static PVs remain in the Kubernetes API, available for future use. Once a PVC is made, the PV provides the requesting pod with storage.

• Dynamic provisioning—if a static PV does not match a newly made PVC, the cluster tries to automatically provision a PV created according to the required storage class. Dynamic provisioning can work only if the storage classes are specified and configured in advance. Additionally, administrators need to enable the access controller for DefaultStorageClass in the API Server.

Binding

Here is how binding works in Kubernetes:

• A user creates a PVC—defining the requested storage size and access mode.

• The master matches resources—using a control loop, the master monitors new PVCs and looks for matching PVs. If matching PVs exist, the master binds the PV and PVC. Alternatively, it can also bind a dynamically-provisioned PV with the new PVC.

This process works to ensure that each request is fulfilled. As a result, users may receive a volume that exceeds their request.

Here are the two possible outcomes of this process:

• The matching PV and PVC are bound and remain together indefinitely.

• A PVC with no matching PV remains unbound indefinitely. It may be bound only when a matching PV is made available.

Using

Here is what you should know about using PVCs:

• When a pod attempts to use a PVC as a volume, the cluster locates the bound PV and mounts it to the pod.

• A PVC user can specify the desired access method. If the volume supports multiple access, the user can specify the desired mode when using a PVC as a volume.

• Once a user takes a bound PVC, the bound PV belongs to this user only. The user can then access the PV via the PVC in the pod’s storage volume.

Reclaiming

A user that no longer needs a volume can delete this PVC object from the API that enables reclamation of the storage resource. A PV has a reclaim policy that defines what the cluster can do with the volume after the claim is released. Currently, volumes can either be deleted or retained.

Kubernetes Persistent Storage in the Cloud

AWS

Amazon Web Services (AWS) offers a cloud-based Elastic Block Store service designed for persistent storage. The service offers high durability by replicating EBS volumes automatically inside their allocated availability zone (AZ).

You can use EBS volumes as persistent storage for your Kubernetes clusters. The service lets you provision EBS volumes in ReadWriteOnce access mode, mounting them on one node at a time. AWS offers the following types of volumes:

• HDD volumes—available as throughput optimized (st1) and cold (sc1).

• SSD volumes—available as provisioned IOPS (io1) and general purpose (gp2).

Each storage class offers different configurations and pricing structures.

Azure

Kubernetes supports two Azure cloud storage services:

• Azure Files—a fully managed file shares service. It supports all three access modes, including ReadWriteOnce, ReadWriteMany, and ReadOnlyMany.

• Azure Managed Disks—offers durable, cloud-based block storage. The service supports only one access mode—ReadWriteOnce.

You can use Azure Managed Disks and Azure Files as persistent storage for your Kubernetes clusters. Additionally, you can use Azure Disks to create a Kubernetes DataDisk resource.

Google Cloud

Google Cloud offers persistent disks that you can use as persistent volumes for your Kubernetes clusters. The service supports two access modes:

• ReadOnlyMany—these volumes can be attached to multiple nodes.

• ReadWriteOnce—these volumes can be attached to a single node.

Google Cloud offers the following types of persistent disks:

• Standard PD—offers HDD and standard throughput. It is generally considered the most cost-effective option. You can use Standard PD for scale-out analytics with Kafka and Hadoop and cost-sensitive applications.

• Balanced PD—offers SSD at the best rate per GB. It is ideal for regular workloads like web serving, line of business apps, and boot disks.

• Performance PD—offers SSD at the best rate per input/output operations per second (IOPS). It is ideal for performance-sensitive applications like databases, scale-out analytics, and caches.

• Extreme PD—offers SSD optimized for applications with high-performance requirements, such as SAP HANA and Oracle.

• Local SSD—offers very low latency. It is ideal for hot caches offering the best performance for analytics or media rendering.

Conclusion

In this article I explained the basics of Kubernetes persistent storage and covered key concepts including:

• Persistent Volumes (PVC)

• Persistent Volume Claims (PVC) and the binding process

• Kubernetes persistent storage in popular public clouds - AWS, Azure, and Google Cloud

I hope this will be useful as you take your first steps in Kubernetes storage management.