The cloud infrastructure market is estimated to be worth some $120bn, according to analyst house Forrester, and cloud storage accounts for a growing percentage of the enterprise storage market as it displaces on-premise arrays and appliances.
An emerging trend in cloud computing is hybrid cloud storage . Hybrid architectures enable organisations to use the same storage infrastructure in both public and on-premise. Hybrid allows data to seamlessly move between cloud storage, local datacentres, or between cloud storage providers.
With the increasing use of object storage technology, and widespread adoption of Amazon Web Service (AWS) S3 app programming interfaces (APIs), hybrid data storage is also gaining ground in enterprises.
This trend was accelerated by the need for organisations that provide remote workers with greater access to data during pandemics.
“Traditional corporate files servers behind firewalls, and VPNs[virtual private networks] can be cumbersome to use when your workforce are mostly remote or spread around the globe,” said Brent Ellis, senior analyst at Forrester.
“A hybrid-cloud server allows you to get files to people. It is easy to add storage, as you can use the cloud to expand your capacity.
Cloud storage is not without its challenges. These can vary depending on the application suite used, IT maturity, and workload. These are just a few of the common obstacles, but there are also some solutions.
Cloud is not suitable for all applications (or their storage)
Not all applications can run in the cloud. It could be that an application is compatible with the cloud, and it will run. However, the performance of cloud storage is not up to the business’s requirements.
Some applications require Posix-compliant storage to work. Posix-compliant storage systems that are not block or file format compatible exist. Object storage is not compatible with Posix.
Cloud providers have succeeded in moving file-based storage from the local hard drive to the cloud. This works for unstructured data as well as applications like archiving. Performance is good with this connection to the cloud. But for high-performance applications that rely on block storage, there is still a gap.
” For file, there are performance and management advantages to hybrid cloud storage when dealing with a distributed workforce,” Ellis says.
” Hybrid solutions are flexible and allow for better performance management. Block is more about letting the application layer use cloud resources, and storage that is flexible. However, performance management must be planned.
Block storage is still the best option. You can customize your arrays to fit your applications. Block storage is best for database-based applications.
The solution is to have some workloads kept on-premise and put other, less-demanding ones in the cloud. There are technologies that enable workloads to be moved between the cloud and on-premise.
One refers to Amazon S3 on Outposts. This is where AWS supplies object storage (on-premise S3). Ellis also mentions Project Alpine, a Dell initiative that can transfer a Power Store volume from cloud to on-premise hardware.
” These hybrid systems must be built in datacentres with high-speed interconnects to the cloud to achieve that performance,” he said. “The benefit is to use compute or serverless functions from multiple cloud providers, and they must be able to abstract storage across multiple environments including on-premise.”
Hybrid cloud is more complex – and harder to manage
Hybrid storage system can also lead to complexity. One area that can be complicated is the management layer. Some industry observers believe hybrid storage systems are more complex than all-cloud or all-on-premise systems.
Administrators will have to deal at the most with two management interfaces.
” To take advantage some of the latest technologies, you often need to spend effort on transforming applications so they can take advantage,” says Tom Bragg (consultants KPMG), an IT infrastructure architect.
” This means you need to find people who are still able to write applications. You also have to store all the source code that you have… It is not always easy. Data storage is always going to be a problem.
The answer is to find technology that works, out of the box, across both environments. These include S3 Outposts and IBM Cloud Object Storage. NetApp’s StorageGrid, Cloudian, Cloudian, Cloudian, Cloudian, Cloudian, and other technologies. One factor in IT managers’ favour is that most use the AWS S3 APIs as building blocks.
“Hybrid storage and multicloud storage to meet S3 compliance storage requirements is a simple solution,” Ellis from Forrester. It is much easier to transfer workloads from the cloud to on-premise applications by having a consistent object storage layer.
Hybrid cloud storage can cost more
Cloud storage can be expensive. Adding cloud capabilities to existing storage systems can increase costs dramatically.
Cloud offers benefits over the upfront cost of on-premises systems. However, firms may have to pay fees for both long-term storage and data egress.
Applications that go to the cloud may incur unanticipated data storage fees and/or costs for bringing it back in-house.
These fees can be avoided by firms by modeling their likely usage and costs and designing their cloud infrastructure to minimize egress costs.
” People tend to keep their data in one cloud or keep it in one cloud because they have the option. Bragg from KPMG says that it is expensive to move data and exfiltrate it each time. These costs can be offset by lower licensing fees and database administration expenses.
Cloud-native applications usually need object storage
The trend towards cloud-native apps is one of the strongest drivers for hybrid cloud and hybrid storage. Although it can be difficult to port traditional applications to the cloud, cloud technologies can be used on-premise.
This is still an area in development. However, if an app or workflow is capable of running in the cloud, it may also be run on-premise, cloud architecture or can move between cloud providers.
” Organisations need to find common ground across silos to make the most hybrid and multicloud. This means using Kubernetes, container-based apps, and Kubernetes, to create applications that can run in every cloud. Patrick Smith is the field chief technology officer (CTO), for Europe, the Middle East, and Africa at Pure Storage.
Containerisation does not solve the storage problem itself, but it gives developers a path to object storage. Ellis notes that hybrid storage should be cheaper and easier to use if there is competition among vendors.
A hybrid future?
It remains to be determined if containerisation will resolve all issues associated with hybrid architectures. According to Bragg at KPMG, most businesses want to migrate to the cloud. However, this is not yet the case. Hybrid can be a bridge to this future.
“You’ve accepted that you’ve got a five- to 10-year roadmap on-premise, so you have to make a decision about whether you want to provide cloud-like services to your on-premise, hosted users,” he says.
“This conversation will include object storage. Are you willing to offer that capability on-premise so people can put a steppingstone into the cloud?