Real-World Financial Services Cloud

Cloud computing has been made very real today.

As I subtly alluded-to in March, The New York Stock Exchange (NYSE Euronext) today launched their Capital Markets Community Platform (CMCP) along with partners EMC and VMware. It's essentially a high-performance, low-latency, special-purpose cloud IaaS, replete with customers and roadmap. Register Coverage  FIN Alternatives Coverage

This is a very notable event for a few reasons:

• Cloud is not a commodity: Unlike general-purpose public clouds, NYSE has constructed a high-performance, low-latency infrastructure to meet the specific needs of trading firms. From these perspectives alone, use of a public cloud (AMZN, RAX, etc.) would never meet the stringent performance requirements. My belief is that we'll see even more of these industry-specific clouds arise. Differentiators will likely vary based on needs for performance, privacy, security, scale, etc.
• Cloud is highly reliable:  A lingering question has been whether the cloud - and associated automation controls - was reliable enough for mission-critical applications. NYSE is no stranger to Financial-Markets levels of reliability, and has clearly taken great pains to ensure that their experience carries-over to the CMCP.
• Cloud is highly secure: Ditto to above. the CMCP is accessible to customers only via a highly-secure network and only to pre-validated users.
• Cloud enables new forms of business: For me, this is the most exciting aspect. NYSE's cloud now allows small firms (picture 3 hedge fund managers and their dog in a garage) to take advantage of enterprise-grade hardware and data... say to test and run new trading algorithms. Access to resources such as this would have been far outside of the reaches of the small firm.
• The Cloud + Big Data story is real: What's also nifty about NYSE's implementation plan is that it allows users to create DB's on demand, and will allow users to access massive data in the form of market play-backs.  This DBaaS will obviate the need for tenants to replicate TB or even PB of data as they test algorithms against historic market data.

NYSE's partnered with EMC and VMware to construct the cloud, and VMW has also posted an excellent Blog on the topic. A few excerpts:

"So, why is this better, and why is NYSE Technologies the right organization to deliver? For hedge funds and other buy-side firms, their value isn't in integrating compute, storage, networks and security -- it's in analytics, trading strategies, algorithms, application strategies and other proprietary expertise. The NYSE service means those IT organizations no longer have to struggle with integrating data dumps and feeds into their infrastructure and operations. Trade execution speed can be critical, so physical location and proximity to the market matters. NYSE's experience in operating large scale, mission-critical VMware-based infrastructure -- the NYSE and Euronext exchanges -- is unquestionable....

"...NYSE represents an alternative cloud future: one that contains a vibrant ecosystem of clouds, both internal IT departments and external cloud service providers, with unique understanding and focus on customer needs, married with the ability to deliver through scalable, on-demand and trustworthy IT services. What internal IT organizations and cloud providers like NYSE share is a rejection of the concept of an inflexible cloud monoculture. Instead, they choose to build high performance, secure and scalable infrastructure because it meets critical business needs. They obsessively focus on value delivered to the customer and never confuse that with cost of service.

And that's it. Cloud is now about Value, even more than it has ever been about cost reduction.

A Community Cloud: Real-World Example

When I first heard the term "Community Cloud" I shuddered. I thought: Just what we need... another cloud definition.

But I had a peek at one yesterday speaking with an established services customer (who must remain anonymous for the moment). They got their start building a co-location facility for companies in their specialized and highly-regulated industry.  But it became obvious that they could add more value as a service provider than just supplying a cement slab, cooling and electrical outlets.

So they've set out to create a raw cloud IaaS infrastructure, but with some attributes that are specific to the community/ecosystem that they serve:

• Security: Access to the cloud is granted only after a trusted validation of identity (required by regulating bodies) - and certain out-of-band management functions can only be made over hardware VPNs.
• Availability: Cloud resources are available at roughly a five-9's level (or better) including complete fail-over and DR sites - this is uber-Enterprise-Grade availability.
• Performance: Because of the specialized industry, the processing and networking performance of the cloud is optimized for high transaction rates and extremely low-latency.

Most other cloud properties, such as elasticity and metering are as you would expect.

Because of the special attributes, the company aims to become a special-purpose Cloud Service Provider to its industry - something that a generic AWS, Google or Rackspace could never be. And many other firms in the industry -- large and small -- will likely find both economic and performance advantages to host in its infrastructure.

Then, things really got interesting...

In addition to the raw IaaS they'll provide, they also plan to provide a special-purpose PaaS to tenants. For example, most clients will tend to use a common set of "Big Data" - ranging in size from Terabytes to Petabytes. If each tenant maintained their own instance of this data, it would be massively costly, inefficient and complex. So instead, the company will host a single, on-site shared instance of the data, charging for its access and use by users of the cloud. And they expect to offer a wide range of such PaaS services in the future.

What does this example say to me?  That (as many predict) the market may in fact only support a very few number of generic IaaS providers who compete almost solely on cost and economies-of-scale. But, assuming this example is even partially successful, there will be room in the market for countless "community clouds" serving the special needs of enterprises and ecosystems globally.

I'd be interested to know if you're aware of opportunities (or instances) of other real-life specialized community clouds in your area of business. The era of cloud has only just begun.

Cloud Attributes Apply Across the Stack

My “aha” moment here at EMC came during my first week when I was asked to describe the generic attributes of cloud infrastructure. Here I was, in an organization that’s made billions on storage, and I was about to talk about cloud attributes solely from a compute perspective.Was I missing something?

I then realized that I’d always related to storage as a “big, fat, dumb disk in the sky”, and assumed that it was merely subservient to the compute stack.

Well, not exactly.

My re-think was that attributes of Compute, Storage, and yes, Network, all had to be reconsidered in the context of a holistic cloud-based infrastructure.

Cloud Attributes:

Most will agree that the following attributes describe the operational profile of a generic cloud: (HT to IDC)

• Shared, standard service. Built for a market (public), not a single customer
• Solution packaged. A “turnkey” offering, integrates required resources
• Self-service. Admin, provisioning; may require some “onboarding” support
• Elastic scaling. Dynamic and fine grained
• Usage-based pricing. Supported by service metering
• Accessible via the Internet. Ubiquitous (authorized) network access
• Standard UI technologies. Browsers, RIA clients, and underlying technologies
• Published service interface/API. Web services and other common Internet APIs

 I’ll add a few functional attributes as well:

• Consolidation: ability to make optimal use of lower-level resources
• Automation: ability to self-configure to provide the required service
• Self-healing/failover: ability to correct for failure with little or no service interruption
• Multi-tenancy, Multi-tiered-SLA: ability of resources to securely house individual services & service-levels across a shared infrastructure
• Global availability: ability to provide a shared service across multiple availability zones

Attributes in a Storage Context

The first assumption most make is that these traits apply exclusively to the compute layer (physical servers, VMs and the like). But pause and consider the storage (and network) facilities need to embody most, too.

But consider this: In a virtualized world, servers are files, and files are just data.

So, when we talk about cloud-related scaling, service migration, server fail-over etc., we must also implicitly speak of managing data dynamics, data replication and data mobility. When we talk about automation, self-service provisioning and service elasticity, we’re implicitly talking about dynamic data/storage provisioning and expansion. When speaking of multi-tenancy and tiered SLAs, we’re also speaking of shared storage facilities performing identical functions in lock-step with the compute facilities.

From a broader perspective , begin to consider implications of global availability and hyper-scale. The terabytes of data that embody virtual servers and their data might need to be migrated to (or duplicated in) multiple hemispheres- not a trivial task from an integrity and latency perspective. We can know (or hope) that the physical servers will be there… but it’s the bits that still have to travel.

The next idea these observations triggered was the need to keep compute, network and storage stacks in lock-step when rolling-out cloud services. The answer (not surprisingly) is converged infrastructure... An approach where the desired cloud attributes are assigned to the 3 stacks simultaneously. More about that in a future Blog.

But I'm now encouraging everyone to view storage of bits in a completely different light – one where the functional and operational attributes of storage must be architected to embrace the core attributes of cloud computing. For without the bits, there can be no servers, no data, and no services. More about that in a future post as well :)