I have long been a fan of Peter Deutsch’s fallacies (btw I’m not alone, Google this AM produced over 22k references) of network/distributed computing, they have served as a set of guiding checkpoints for every distributed system that I have built. What I have found to be missing, however, is a similar set of fallacies/truisms for managing Information while we approach “internet scale” information infrastructure… the information explosion.

Truisms defined by/principles for managing information explosion:

1. no one person/system is capable of managing all data
2. optimizations will be continually applied, but by different vendors, thereby requiring an enterprise to distribute their information architecture
3. information processing is inherently a pipelined process (though fork/join supports parallelism for reduction of latency)
4. these pipeline’s can have “in parallel” replicas so long as sufficient locking is engineered, and compensation models supported
5. locking for a given pipeline should be owned discretely by a single application context (workflow) - though this workflow may be complex, it is stateless upon completion of end state
6. loose coupling / jit integration require coherent, federable, data dictionaries and meta-data/structure maps

Translated to Fallacies… which, agreeing with SGG, I think are way more powerful, and in some cases hilarious.

1. there is one enterprise data architect who is responsible for the master models
2. there is a system who is the authoritative master for a given entity domain
3. there is one vendor involved across the SOA and EIM domain
4. the data models are largely fixed, and the business will not ask for further changes/enhancements to the model
5. data exchange will be based upon XA/2-phase transactional mechanisms to achieve ACID properties (pessimistic transactionality)
6. there will be a singular data dictionary, with complete meta-data for a given entity domain

Additions/Subtractions/debate most wanted!

Technorati Tags: distributed system, information infrastructure, truisms

So there has been much ado from commercial technologists, analysts and press about the emergence of SOA as a silver bullet in the defining of a common Enterprise / Inter-Enterprise Service model, and in an approach to the re-factoring of existing legacy systems. Though there are almost as many approaches to SOA as there are articles defining and re-defining the terms, one of the critical lessons is that SOA will only yield as much agility as the infrastructure upon which it runs.

Coming from a history of grid/utility computing programs has afforded me the benefit of viewing enterprise resources as consistent modelable elements that can create a virtual deployment infrastructure for this next generation of agile enterprise services. What remains critical, and a common theme in my weblog has been the need to better elaborate the systemic requirements of a system - whether represented by a single service interface, or through a composite model, service level objectives remain a difficult to define, more difficult to measure, and virtually impossible to predictably enforce set of goals. A recent read of the Enterprise Grid Alliances “Reference Model” is illuminating; as I think that the industry, through GGF, EGA, DTMF UCWG, SNIA and even eTOM are getting close to a basic taxonomy in which a SOA is deployed (hopefully dynamically) against a Service Oriented Infrastructure (SOI). So what am I really saying? Model Driven Architecture is attempting to establish consistent patterns for the organization of object oriented services, with clear separation of concern, and appropriate compositional granularity. SOA’s are constructed through Directed Acyclic Graphs of these sub-systems that define their inter-relationship (though the complete vocabulary isn’t standardized). Now that we understand the components that make up an enterprise service, we need to determine how to apply these models against infrastructure, and once again MDA can serve us well.

Let’s take our infrastructure (pools of elemental resources) that can also be dynamically assigned, are themselves objects, which can leverage similar model based approaches to their organization we quickly begin to recognize the need for both “extension” - supporting a consistent interface to a domain model, whilst enabling the customization that is a natural byproduct of real-life variability. Models, for me are a composition of patterns that provide layered abstraction to mitigate complexity and support replacement… These models are really purpose built realizations of patterns based upon a set of concrete contexts - a complicated way describing constraint driven pattern application…. I want to remodel my bathroom, to build a plan I need to take existing patterns from some kind of “standard” catalog, and apply them given my constrains… I’m tall, so I want the countertops raised, I’m disabled so I need to extend the natural 18“ on center rule for toilet spacing to 36” for access… you get the picture. By producing models (either abstract or concrete) we can begin to elaborate a design, manifesting all of the complexity within a set of stackable “layers” that ensures both loose coupling/east of replacement, along with the traceability against requirements and constraints.

So finally to VOA or “Virtualization Oriented Architectures”; architectures built with clear layer based / atomicallly separable abstractions… VOAs are Models in which abstraction can be late bound, but coherently orchestrable designs. These designs need to describe the “virtualization” constraints in a way that deployment contexts can be manifest at runtime rather than design time, through a more consistent set of abstraction layers/patterns that reflect the platform layer, service layer that you wish to virtualize. In lay terms… virtualization provides a component and a container that inherently provides the lifecycle management of that component. For VOA to work (as SOA) the functional capabilities as well as the non-functional or systemic capabilities need to be declared at design time, and then contractually provided at runtime.

Technorati Tags: distributed system, soa, virtualizatoin

As many know HP announced it’s intention to buy OpsWare for $1.45B on Monday (7/23/2007). Some seem think that they are just buying into a negative cash flow business. My gut, tells me that OpsWare has had problems because deeper management software integration (OpenView) and HW integration would help them to create multiplicative value in operational efficiency; easing the “we’d buy it if it were just a little more integrated” denial.

So let me just play devils advocate… HP has been investing in virtualization/grid technologies… see HP SFS basically a bundled “storage grid that is Lustre ready, supporting 35+GB/s performance (ether) and 1PB of capacity built from small boxes (hp’s quote: ”3X the bandwidth at half the price of scalable NFS offerings.“)… Sun is trying the same thing with Constellation and their x4500 (thumpers), the challenge that we all have is that vertical scale brings systemness, but on a non-commodity price curve. The commodity/sedimentation / scale out architectures, however have a dirty little secret, as you scale out the complexity of management becomes high, and their provision becomes more rigid because of this… impacting capital arbitrage and schedulable density (fancy ways of saying, I have to lock apps to boxes to ensure that I can manage, instead of allowing apps to be dynamically scheduled to available resources (compute, network and storage).

In fact, what grid technologies do is add systemness to compute fabrics, and are hugely complimentary to existing virtualization / abstraction layers like VMware and Xen. So for me, Opsware is a ”platform“ that can help HP control the migration of their customers from static deployments/manual processes -> static deployments/automated processes -> dynamic deployments/automated processes.

I think that we’ll see other activity soon around BladeLogic from the like of BMC and IBM. The gauntlet has been thrown down!

Technorati Tags: distributed system, hp, network management

Finally…. (thanks John Walton for the heads up)

Longbow XR allows arbitrarily distant InfiniBand fabrics to communicate at full bandwidth through 10Gbits/s Wide Area Networks. The WAN connection is managed out of band, and except for flight time induced latency is transparent to the InfiniBand hardware, stacks, operating systems and applications.
XR achieves flow control by shaping WAN traffic and managing buffer credits to ensure extremely high efficiency bulk data transfers — including RDMAs — making the system a highly effective transport mechanism for very large data sets between geographically separated InfiniBand equipment.

Read More

Technorati Tags: fabrics, infiniband, networks