HumbleBlogger

After 17 years is it too late to fix C++ runtime extensibility?

2009-04-04T23:46:00.000-07:00

The 1992 to 1993 timeframe was pivotal and fateful for C++.

In the '92/'93 time frame I worked on a plugin architecture for Aldus PageMaker, which was coded in C++. PageMaker was built on a C++ OOP framework called VAMP, which assisted its portability between Mac OS and Windows.

So we tried to use the features of C++ to build a plugin architecture. This proved to be very problematic for C++ classes due to the so-called brittle base class problem.

I proceeded to write a paper that was published in journals and that I presented at OOPSLA '93 in a reflection workshop. You can read the pdf format of the paper I presented here:

Time Invariant Virtual Member Function Dispatching For C++ Evolvable Classes

(Page to the bottom and click View or Download)

I also made contact with Bjarne Stroustrup at a Usenix conference in Portland and proceeded to dialog with him for several months, where he championed the issue of dealing with the brittle base class problem on my behalf. (Alas, other issues were deemed more important at that time - for instance, trying to shepard RTTI through approval.)

Microsoft at this time introduced the COM/DCOM system for the Windows platform that was looked on as a viable solution to the problem. C++ could be used as an implementation language for COM via abstract classes used to define COM interfaces.

However, these days developers shun COM/DCOM - and especially ActiveX, that was based on COM. In 1994 I went on to work at Microsoft and used C++ and COM/DCOM there for the remainder of the decade. I came to conclude that the technology combination was a definite dead-end. I refer to that experience here:

Building Effective Enterprise Distributed Software Systems

In contrast to all this early C++ history, Steve Job's company, NeXT, devised a plugin architecture using Objective C during the early 90s, and that was integral to the NeXT Step framework. Today that lives on vibrantly in Mac OS X on Apple's computers and important platforms such as the iPhone.

I submit Objective C enabled solving the plugin problem in a superior manner.

The champions of Java will cite factors such as garbage collection as to why it is more productive than C++. I won't dispute that, but Objective C has not had garbage collection until very recently with 2.0. On the iPhone garbage collection is still not available. None-the-less, the OS X plugin architecture has been entirely viable - due entirely to the merits of Objective C style of OOP vs C++ OOP.

Interestingly, Java has been used to build the most pervasive plugin architectures that exist for any platform or software development community. The Eclipse IDE plugin architecture, which by now is practically legendary, as such architectures go, incorporated a few years ago the Equinox OSGi module management layer. J2EE app servers have supported a plugin architecture for EJBs for a decade. These days all J2EE app servers of note have likewise incorporated OSGi to manage runtime bindable modules. The Spring Framework and its runtime bindable unit of the Spring Bean has grown to exceed J2EE - primarily on the strength of its system for managing the binding in of the Spring Bean.

The Java community is still struggling to define an official module management standard yet even as things stand, the Java platform supports software architectures that incorporate plugin capability more pervasively than any other programming platform.

Despite the weaknesses of Java as a language relative to C#, and that .NET already has a stronger module implementation in its .NET assembly standard, Java is still light years ahead in terms of everyday software systems in wide use that incorporate some form of plugin architecture. Strangely, the Java community doesn't even consciously realize that this is the underpinning of their continued success as an enterprise development platform.

I personally regard the brittle base class problem of C++ to be it's most fatal flaw.

After 17 years have passed since this issue was highlighted to the C++ community, and to the creator of C++, is it now too late to address it?

Distributed OSGi — tilting at windmills

2008-06-15T00:04:00.000-07:00

I've been an enthusiastic champion regarding the advent of OSGi as used for a Java-based enterprise computing modularity standard. However, I have a very definite compartmentalization in mind as to what role OSGi should play — I draw the line at a single running JVM instance. I don't want to use OSGi as a cornerstone for distributed computing modularity.

In a recent eWeek article, though, we see there is a movement afoot to do precisely that:

Distributed OSGi Effort Progresses

This excerpt makes it quite clear what this initiative is all about:

The Distributed OSGi effort also seeks to enable "a service running in one OSGi framework to invoke a service running in another, potentially remote, OSGi framework (meaning a framework in a JVM)," Newcomer wrote. As the current OSGi standard only defines how services talk to each other only within a single JVM, "extensions are needed to allow services to talk with each other across multiple JVMs — thus the requirements for distributed OSGi on which the design is based," he said.

I see such an effort as yet another attempt to reinvent what has already been tried several times before — distributed object computing — by the likes of such technologies as DCOM, CORBA, and even JEE/EJB (ala Session beans and RMI). What do these technologies all have in common? They have fallen into disuse after befuddling and frustrating many a programer (I personally fell on the DCOM sword back in the mid-90's).

Interface Calling Behavior Semantics

I have a by-line that I sometimes sign my blog postings with:

Let me get right to it by lobbing some grenades: I recognize two arch evils in the software universe – synchronous RPC and remoted interfaces of distributed objects.

The sentiment being expressed here is that it is wrong-headed to try to make synchronous method invocation transparent over the network. Yet that is the grand illusion that all these distributed object solutions strive to accomplish.

The problem is that an object interface that may be perfectly usable in a local JVM context — where call chain invocation can be sub-millisecond — will not have the same behavior semantics when invoked over a network connection. The method invocation may take 15 to 50 milliseconds on a good day; or may fail due to low-level transport errors (which never existed when it was being invoked in a local JVM context); or just time out without completing the call; or even never return/complete at all in any acknowledged fashion.

The consuming software code that used a method in a local JVM context has to now be designed to anticipate a wide range of different situations, as the calling contract of the method is radically different depending on the context in which it is being invoked. The advocates of distributed object computing, however, want us to believe in a grand illusion: that modules that were written and proved out for use in a local JVM context can be shifted to be consumed in a distributed context — and where consuming code, presumably, doesn't have to be any the wiser (or at least not very much wiser).

Of course, some may recall JEE/EJB Session beans where methods invoked via RMI had the potential to raise exceptions related to i/o transport errors. The upshot is that one had to design software from the outset to be a consumer of a "distributed" object interface vs. just a consumer of its local interface. Also, it was not long before EJB developers discovered that an object interface that made sense for a local JVM calling context would become a performance liability in a distributed computing context.

To use the interface by its design gave rise to chatty round trips over the network, where, due to the latency/unreliability, the software becomes visibly sluggish. It is most dismaying to see all the enterprise software systems that have sluggish and problematic user interface behavior due to the application being written on a foundation of synchronous use of distributed object interfaces. That distributed object koolaide that folks drank from proved to be spiked heavily with toxic radiator fluid.

In essence, EJB developers found out that an object-oriented approach to software design could not be transparently shifted to a distributed computing context. The OOP software systems that tried to make that leap devolved into a quagmire of issues that had to be battled.

Interface Version Management

On the basis of this one failing alone distributed object computing has been one of the greatest colossal architectural mistakes of the last 15 years in the IT industry. Yet the failings don't stop there — the other equally perplexing obstacle to this undertaking is object interface version management.

I tend to think that the versioning dilemma is perhaps even more insidious than the synchronous distributed method call semantics problem. One encounters the issues of call semantics fairly early on, however, the interface versioning dilemma arises gradually over time, and then mounts up and becomes one of the greatest headaches that one battles in trying to keep deployed distributed software systems coherent.

One of the popular agile OOP developer practices of recent years is frequent re-factoring of code. Indeed, all of the popular IDEs in use are adept in assisting the developer with re-factoring. Re-factoring may well be a good thing in a context where the development team gets to control all the deployment pieces that are impacted by such change. However, in a distributed computing context, which is usually heterogeneous, re-factoring would just be asking for misery.

Making changes to how distributed software systems interact, and where multiple development teams and/or companies are involved, is a process undertaking akin to wisdom tooth extraction (the difficult kind where the dentist has to work for a few hours to break the tooth apart and bring it out in pieces). The simplest of changes can be tedious to negotiate, often politics of some variety intrudes, and it is often challenging to schedule releases to well synchronize with one other so that deployment can occur.

As such, the notion of versioning of distributed object interfaces has been proffered as the means for coping with this. One team can come out with a new and improved interface to an existing object and deploy it unilaterally. Other parties that devise software that consumes the interface of said object, can catch up to the new interface as they are able. In the meantime the older interface remains in place so that existing deployed software keeps working.

On paper versioning looks like a workable approach — the significant distributed object solutions have all had provision for versioning interfaces. In practice it can even be done a few times. However, for large, complex enterprise software systems, maintaining interface versions gets to be burdensome. One of the reasons is that by their very nature object interfaces are very brittle. The information state that is exchanged tends to be very explicitly coupled to the interface signature. It can be hard to significantly (or meaningfully) evolve the software implementation of an object without having impact to the object's interface. Once that happens, the interface has to be versioned — and a sense of dread then sets in.

OOP Does Not Work Well In Distributed Context

As to distributed object computing, quagmire is the operative word here. Quite simply OOP does not really work in a distributed computing context — there are too many difficult issues that entangle for it to be worth the while.

It is fascinating to see new generations of software engineers getting lured into re-inventing distributed object computing over and over and over again. And a lot of the same computer industry corporate players get right behind these endeavors every time. These kinds of systems become complex and grandiose — thus they seem to be excellent sticky fly traps for luring in developers. Think of the legions of developers (and their companies) that have floundered on DCOM, CORBA, JEE/EJB, WS-* (death star) — and now lets add Distributed OSGi. Distributed object computing is our industry's Don Quixote tilting at windmills endeavor.

Asynchronous Messaging and Loose-Coupling Message Format Techniques

So what is the alternative?

When it comes to distributed computing interactions, try following these guidelines:

Design the software from the outset around asynchronous interactions. (Retrofitting synchronous software designs to a distributed context is a doomed undertaking that will yield pathetic/problematic results.)
Prefer messaging to RPC or RMI style interface invocation
Attempt to use messaging formats that are intrinsically non-brittle. If designed with forethought, messaging formats can later be enhanced without impacting existing deployed software systems. (The entire matter of versioning interfaces can be dodged.)
Build in robust handling (and even auto-recovery) of transport related failure situations.
Never let the user interface become unresponsive due to transport sluggishness or failure situations. A user interface needs to remain responsive even when over-the-wire operations are coughing and puking. (So distributed interaction I/O always needs to be done asynchronously to the application's GUI thread.)
Keep transport error handling orthogonal to the semantics of messaging interactions. (Don't handle transport error and recovery at every place in the application where a messaging interaction is being done. Centralize that transport error handling code to one place and do it very well just one time.)

A key point to appreciate is that transport error handling and recovery is very different from the matter of business domain logic error processing. The two should not be muddied together. A lot of software written around PRC or RMI style interface method invocation does exactly that, though. Messaging-centric solutions usually permit a single transport error handler to be registered such that this need be coded just once. The rest of the application code can then concentrate on business domain logic processing.

AJAX and Flex Remoting

A messaging approach is a sound basis for designing distributed application architecture — especially when one does not control all the end-points. More recently I have been designing architecture for Flex-based web RIA applications. In these apps, the client uses Flex asynchronous remote method invocation to access services on the server. Adobe's BlazeDS is embedded in the server to facilitate: remoting calls, marshaling objects between ActionScript3 and Java, message push to the client, and bridging to a JMS message broker.

You may think that I'm not exactly following my own advice. However, there are special circumstances at play:

Flex I/O classes support asynchronous invocation, so the operation does not block the main GUI thread of the app.
Flex I/O classes invoke closures to process return results; also, a fault closure can be supplied to handle transport related errors. Consequently a programmer can write one very robust fault handling closure and reuse it in all I/O operations. Thus Flex does an excellent job of segregating business logic processing from transport-related error handling.
Flex client .swf files are bundled together with their Java services into the same .war deployment archive. Consequently, the client-tier and the server-tier are always delivered together and thus will not drift out of version compliance.

The last point is worth further remark: The way the two distributed end-point tiers are being bound together into a single deployment unit makes for a situation that is nearly like delivering an old-school monolithic application. Flex supports Automation classes so that tools, such as RIATest, can be used to automate testing client UI. Consequently the client can be scripted to regression test against service call interactions. Thus deployment unit is the subject of due QA attention and even though the two tiers are not subjected to the same compiler, at least they actually get tested together prior to releasing.

If the service call interfaces are refactored, then the client can be refactored at the same time. Typically this is even being done within the same IDE (such as Eclipse with the Flex Builder plugin). The Flex code and the Java code each have refactoring support. Flex unit test could then be used within the development context to verify call interface validity.

Google GWT applications have similar characteristic where asynchronous method invocation is supported for invoking services on the server tier. Client tier java code and services jave code is developed co-jointly and can be packaged into a single deployment unit.

AJAX web applications may be another case where the client tier and the server tier are often deployed together.

Conclusion

So the take aways from this discussion are:

If you can't control both end-points stick with messaging and loose-coupled message format design. Be very mindful of the versioning dilemma from the outset and plan with forethought. The best outcome is to be able to evolve message formats without breaking end-points that have not yet been upgraded. Try very hard to dodge the burden of version management of installed end-points.
If you can deliver both end-points from a common deployment unit, then method invocation of remote object interfaces can be okay. However, stick with the technologies that support asynchronous I/O. Separation of the concerns of business logic processing on return results from transport fault handling is the ideal.

Flex Async I/O vs Java and C# Explicit Threading

2008-05-18T10:40:00.000-07:00

Explicit Threading for Async Behaviors (Java/C#)

With Java and C# .NET developers, these folks are used to doing asynchronous operations on threads they explicitly create and manage, and then use the utility mechanisms that both Swing and C# .NET each provide to marshal any data to the main GUI thread.

Flex Async I/O

With ActionScript3/MXML based Flex apps, asynchronous operations revolve around doing I/O service calls or messaging interactions with the server-side.

So all the various I/O classes of the Flex SDK can operate in an asyncronous manner. For instance, the HttpService class has a send() method. Invoke send() and it will execute asychronously to the calling thread (which for Flex, will always be the main GUI thread of the app).

ActionScript3 Closures coupled with Flex Async I/O

To handle the result (or any error condition), you can use the closure feature of ActionScript3 language to supply a closure block that will process any result that is later asynchronously returned. The Flex architecture insures that your closure code that is invoked to process the result will actually execute on the context of the main GUI thread. Hence that code can safely interact with the state of Flex GUI objects.

A closure can be supplied to handle a successful business logic result, a business logic failure condition that is perhaps returned to the client - and a separate closure can be supplied to handle faults. Faults would occur because of, say, transport related failtures, etc. - such as not being able to connect to the destination server, I/O timeout, underlying socket errors, blah, blah, blah.

Closures are a very natural and intuitive means to program asynchronous class APIs.

Flex approach is a simpler programming model

The net result is that Flex architecture presents a simpler single-threaded programming model to the developer, while asyncrhonous I/O programming coupled with the language feature of closures, provides a means to create apps that are just as fluid and responsive to the user as are Java Swing or C# .NET WinForms apps. The user can continue to interact with the UI of the app, a progress indicator can be presented, and a cancel button that acutally works can be presented to the user.

So some of the same things can be accomplished in a Flex app but with a better and easier programming model than the explicit thread programming of Java or C# .NET.

Client-side MVC Pattern ala Flex

Just to put a fine point on this subject matter of Flex asynchronous style of programming, what a real Flex application will typically do is implement some variation of the MVC pattern (on the Flex client tier - not the server-side tier).

So the closure that processes the result of some asynchronous I/O service call operation (or a message pushed from the server-side via the Comet Pattern - ala BlazeDS), will typically use the result data to go and modify the state of an object that resides in the client. That object would be some aspect of the data state that constitutes the Model from the MVC pattern.

Properties, Events, Declarative Data Binding

Views onto the Model can utilize the AC3/MXML language features of properties, events, and declarative data-binding to couple interactions with the Model.

A Model object can expose its state as properties. When any I/O closure code modifies the state of Model objects, events on their properties will fire. Views can bind to these Model object properties such that Views will update themselves whenever the underlying Model objects change due to said asynchronous I/O activity.

Views can use a Controller to deal with feeding view changes (which are usually due to user interactions) into the underlying Model objects. Data validation will likely tend to reside in the View code. So View/Controller interaction will concern itself more with feeding user-driven changes in View state (that have been validated) back into underlying Model objects.

Multiple Views onto single Model

One upshot of the MVC pattern is that it now becomes straightforward to implement multiple Views onto the same underlying Model objects. Also, View code gets completely decoupled from I/O plumbing code, as that becomes partitioned into a separate aspect of the client app architecture.

Additional UI Considerations

Views may still need to be designed to say, disable a submit button after it's been pressed (so the user can't click it by accident or intent multiple times). Events could be used to signal to the View when the submit button can be safely enabled again. Likewise, there is the View programming involved for such things as showing progress indicators and/or cancel buttons when asynchronous I/O operations are actively being processed.

Is Flex RIA suitable for content-heavy web sites?

2008-05-11T10:01:00.000-07:00

Is Flex RIA only suitable for business application interfaces, or apps like Buzzword (a highly functional word processor)?

Often it's believed that a web RIA approach is perhaps not so optimal for content heavy web sites (the blogging and social web sites, etc).

However, there are two aspects of Flex that could be leveraged to integrate content and yield a more dynamic UI.

First is that a Flex app can be built like a portal. A Flex form can dynamically load other Flex forms in a nestable like manner. The sub forms loaded can be determined by logic that executes at runtime. The loadable modules can be built as totally standalone components. (Via OSGi, is possible to even bundle these Flex form modules with their own respective Java service layer beans.) They can bind into the hosting portal environment using Flex properties, events, and declarative data binding features. So the coupling layer can be well abstracted so as to not be overly brittle.
Second, one can incorporate a popular iFrame Flex control into a Flex form, which, of course, the iFrame can be used to load HTML/JavaScript content. So highly dynamic web content can be integrated into the UI of a Flex RIA application.
Thirdly, Adobe AIR has WebKit HTML engine built-in, so an AIR app can also readily make use of HTML content in an integrated fashion.
Fourthly, there are already mechanisms that can be hosted in Apache httpd and IIS that will compile Flex forms from source code on the fly. So some aspects of even the Flex RIA could be generated dynamically at runtime. Currently these mechanisms are intended for development phase purposes. The compilation is not considered fast enough for production sites. However, in time this will likely improve.

(Sorry for going a couple of bullet points over.)

In the meantime, my preferred approach is Flex form modularization and Flex control iFrame for embedding HTML content.

OSGi on the Server Side and the matter of the WOO

2008-01-25T17:28:00.000-08:00

With the advent of a SOA approach to building enterprise applications, it is apparent that there is a need of a better modularity solution for Java server-side programming. Is 2008 the year for the ascendency of OSGi?

An Introduction to OSGi on the Server Side

No-Fluff-Just-Stuff founder, Jay Zimmerman, likes to refer to what he calls the WOO factor - window of opportunity - when talking about the dynamics of our software development industry.

For me, Adobe Flex is a case in point of the WOO factor that he talks about. It's a technology that was ready and in position at the right time to become a dominant RIA solution. In contrast Java was caught with nothing to show in the RIA space. It missed the WOO.

Right now, I have an engaged server-side project that will be built on OSGi. OSGi is here today, has a proven track record (Eclipse), and solves the problems of project size scalability that we need to grapple with over product life time.

Once again, just as EJB3 missed the WOO and has been eclipsed by Spring Framework, whatever standard that Sun will ultimately back via the JSR process will miss the WOO.

Why am I writing this as though I'm lamenting a sad development? I guess that's a good question. There's really nothing to be downcast about. Great software solutions on the Java platform exist outside of JSR land. Indeed, maybe the very best software used in Java circles has been the non JSR stuff.

In coming to the Java community I've never had a problem sidestepping the nonsense being peddled and instead adopted what made sense. With JEE I completely ignored session and entity beans and just used JMS MDBs and Hibernate. So in 2003/2004/2005 I built a distributed software system on principals that the SOA dudes are now all crowing about.

Building Effective Enterprise Distributed Software Systems
data-driven communication vs behavior-driven communication

More recently I've completely ignored MVC web frameworks on the server-side and instead went with RIA + SOA where MVC is completely done on the client-side. (Doing MVC on the server-side in defiance of the Fallacies of Distributed Computing was wrong headed from the get go.)

Web frameworks peaking toward obsolescence

So here we stand today poised to fish or cut bait on this matter of large scale SOA-based development on the server-side - and we badly need a modularity solution for Java.

This is the moment of the WOO and here stands OSGi.

Review of ActiveObjects concept

2007-09-22T13:24:00.000-07:00

Below is a review of Daniel Spielwak's ActiveObject for Java, as presented in an article on JavaLobby. Refer to the link to reference the original article.

Dynamic proxy vs byte code enhancement

ActiveObjects is interesting but could be improved by being implemented using byte code enhancement tools instead of Java dynamic proxies. Ruby, et al, are fine with taking the hit of dynamic binding overhead (people expect such overhead with scripting approach, as it's a conscious trade-off with respect to benefit of rapid development). Java is a compiled language, though, where performance is emphasized (and prized) for the purpose of building industrial strength solutions for enterprise class applications.

The author's phobia regarding byte code enhancement only serves to undercut his credibility. Byte code enhancement is well proven as many systems have been in wide use for several years now. In years of working with such tools, libraries, frameworks, languages (AspectJ), etc, byte code enhancement has never once been the cause of any issue or point of grief. Others have like experience or else the forums would be full of complaints.

Convention over Configuration

Convention over configuration is a utopian concept given that in practice enterprise development has to give sway to schema that is legacy and/or dictated by DBAs (and for good reason as relational database schemas need to be designed to perform well - not accommodate idealized object-oriented design).

In the end, for ActiveObjects to be realistically useful, there will need to be a way to use things like annotations in order to map the object perspective to the underlying database perspective. Or else the ActiveObjects entity interfaces could always be strictly generated by a tool that takes the database schema as its input (but event then there will be things that need to be customized via mapping overrides, etc.).

Creating Java interfaces for entities as the starting point only works out for sample apps, the simple Ruby-esque contacts CRUD app, or the occasional (but highly infrequent) green-field situation. Yet even there, a serious app of some complexity can get in trouble if database scheme is directly based on idealized object-oriented design. There are a lot of special things one does in the database to make operations perform well that would never have come from generating off of the object-oriented design of entities. Plus a pure, idealized object-oriented design could lead to pathological constructs in the database - from a relational database perspective.

When in Rome, do as the Romans. When living in the RDMS, do as the RDMS does things and life will be more pleasant.

Lazy-Loading

Lazy-loading seems like a fine idea from a naive (inexperienced) Java programmer's perspective, but in practice one has to work to minimize discrete trips to the database - or else will get thrashed on performance. Trying to always default to a simple heuristic, such as lazy-loading, is another concept that falls down in practice.

Business Logic in Entity Interfaces

Placing business logic methods into entities is pretty much a bad idea.

Often times it is desirable to convey entities over the wire (90% to 98% of the time) - that is, between the middle-tier and client-tier of 3-tier or n-tier distributed applications.

As such, entities need to be kept stripped down to pure data only, ala DTO (data transfer objects).

One of the very worst sins that one can commit is to pollute a distributed application with the anti-pattern of "distributed object interfaces". Attempting object-oriented programming that transparently spans across the network is discomfiting as the network is neither reliable nor deterministic.

Sticking with DTO approach to entities and then create services that operate on or with DTOs avoids such morass.
By tossing objects with behaviour and going strictly data-only, there are then some great enterprise messaging patterns that become possible. Those patterns will more than make up for the abandonment of OOP over the network.

The problem with OOP and network is that OOP as we understand it today is build on synchronous assumptions of behaviour interactions. Synchronous operations and the network mix like oil and water - bad combination. The second (or is it the first?) big problem with OOP over the network is tight coupling. Tight coupling is bad news for distributed application. Either one of these issues by itself is sufficient to doom this concept.

Summary

ActiveObjects needs to have the perspective of another five to seven years experience developing real-world enterprise applications.

The shear rapid ability to write working code, ala Ruby and RoR, focuses on a very narrow part of the overall problem spectrum. When you concentrate so much on this part of the equation and neglect/ignore the others, then the end result is something of limited applicability.

The serious-minded tools/solutions that exist out there have endeavoured to address the wider range of issues that professional developers (at least those in the enterprise arena) confront.

XML data "duck typing"

2006-12-27T20:46:00.000-08:00

What follows is a so-called best practice technique that I've gravitated into as a means to better facilitate the evolvability of my distributed software systems. This was originally posted in the follow-on discussion of a recent Ted Neward article Java, .NET, But Why Together?

One of the most paramount concerns in working with enterprise distributed software systems is the issue of tight coupling between nodes. This particular issue is one reason I prefer messaging vs invoking methods on a remote interface. Interface binding leads to very tight coupling. Tight coupling leads to difficulties in attempting to evolve a distributed system over time. In practice it is seldom feasible to rev tightly coupled nodes in lock step with one another. And the practice of versioning interfaces eventually becomes too burdensome if attempted pervasively.

When messaging is applied with certain tactics it can bring about some significant relief on this front.

Now as I was first starting out with my JMS distributed system development, I was conveying relatively complex XML documents around. These were being described in XSD and then tools like JAXB for Java and XsdObjectGen for .NET were used to generate suitable marshaling code. The documents could then be marshaled into .NET or Java object graphs with ease. This works pretty well (though Ted Neward can speak to some significant limitations and caveats) and the nature of XML Schema, as described by XSD, makes it possible to evolve document schema over time while not disrupting existing code that binds intimately to the resulting object graphs. (Yet one may sometimes still need to regenerate the code when the XSD changes which can require updating deployed code.)

However, I eventually drifted away from this practice and now use an approach that I dub XML data duck typing - a play on the term duck typing that Dave Thomas is well known for using in his Ruby talks.

Basically I was finding that processing nodes tended to have interest in subset information as gleaned from XML documents. This was just a natural evolutionary outcome of organic expansion/enhancement of the distributed software systems. Indeed it was very advantageous to encourage this tendency (a single message could have multiple nuances of effect based on what node processes it).

So I now write components such that they just use XPath to glean the information that they care about from XML documents. I have a technique and a few helper methods that make this a very easy and systematic approach. The upshot of this, though, is that a given node in the distributed system could care less how radically an XML document is evolving over time so long as the info it cares about can continue to be successfully gleaned. If it looks like a duck and quacks like a duck then...

Dropping the code generation tools and just going with XPath-based XML data duck typing has been a breath of fresh air as it breathed a new level of loose coupling into the distributed systems.

BTW, Ted Neward is all over this entire subject matter in a talk he is giving at developer conferences that he dubs XML Services. You can catch him speaking on this at both Java and .NET conferences.

In my distributed application architecture, there isn't really much OOP-think that goes on. Is mostly topology, data flow, event flow, with a lot of emphasis on filtering, routing, bridging, and transformation. OOP is pretty much just relegated to the internal design of distributed components.

Building Effective Enterprise Distributed Software Systems

2006-08-27T09:59:00.000-07:00

data-driven communication vs behavior-driven communication

by Roger Voss, Aug. 26, 2006

Seeds of Doubt

Way back in the '90s, while working at Microsoft, I built middle-ware systems using C++ and DCOM. At one point I redid some of a system with MSMQ messaging and had an ah-ha experience. MSMQ is not the most robust MOM feature-wise, but it was enough for me to see that things can go better with a pure data-centric, asynchronous messaging approach vs. behavior-centric, distributed object remote interfaces (at the time I was becoming very disenchanted with DCOM especially, but also COM in general).

A Home-Brew App Server Points The Way

A few years later on I build a custom-designed, messaging-based, middle-ware application server written in Java during a couple of months of being laid off. (The insurance company I was involved with then decided to cut back on its experimentation with Internet innovations as a cost control measure as due to 9-11 aftermath. I continued to do some R&D, though, in hopes of landing back with the original entrepreneurs of the endeavor.)

The system I built during that brief sabbatical was designed to hook into a user's web browser interactions with the server-side and mirror everything to an agent in a call center - a co-browse implementation. It was essentially a specialized web proxy server intended to reside on the data center side sitting in front of conventional web sites. Design-wise it was sort of a JEE app server in microcosm, though anchored around bi-directional messaging (as opposed to EJBs sporting remotely accessible interfaces). At the time I had had no involvement with J2EE and was still relatively new to Java. It was later that I came to see some of the parallels in what I built vs. J2EE. The similarity was thematic, of course, as certainly J2EE was vastly more intricate and comprehensive.

So I landed in a totally different industry, yet a distributed application project I got assigned to needed the event-driven capabilities of fully symmetric bi-directional messaging and a middle-tier application server. The system I had built was entirely suitable and so I incorporated it with modest adaptation. The application went into production and enjoyed splendid success.

Becoming Enterprise Robust

Now my company had grand long term ambitions for this initial foray. The middle-tier app server would need to accommodate an ever-growing array of applications - some closely related, some distinct. Event-driven distributed design would be a constant theme in all cases. It would also need to support clustering for high availability. Extensive capabilities for monitoring would soon become essential so all these various apps could be managed. Ease of configuration and deployment would be significant concerns as well as this software would be deployed into multiple customer sites. In short, it needed to become a true enterprise class software system.

I was aware of the EJB 2.0/2.1 spec which incorporated Message Driven Beans for enabling easy use of JMS. In looking at J2EE I directly homed in on JMS - from my point of view messaging was the star attraction. After some months of experimentation and product evaluation, the next release of my software system replaced my home-brew app server with a J2EE app server and an enterprise-capable JMS MOM - setting the stage for several years of successful organic growth of said system that's still going strong. I finally got the chance to build the kind of distributed enterprise software that was just a gleam in the eye back in my Microsoft days. (Incidentally, I had a brief diversion of attempting to build a somewhat similar system using ASP.NET. That experience is detailed in this blog post: ASP.NET vs J2EE)

My uptake of J2EE, in retrospect, was rather unorthodox. I used MDBs near exclusively and then also incorporated JMX mbean services to do special things like device control or a UDP-based JMS auto-discovery service (mbean services gave me the latitude to do my own thread pools and device level interactions). I stayed away completely from session beans (I couldn't believe that remoting a component interface could be more convoluted than DCOM). The servlet engine was mostly used as a JMX console viewer, as otherwise I seldom had any use for servlets. Having co-designed and participated in a couple of ORM implementations while at Microsoft, I naturally gravitated to using the Hibernate ORM and consequently shunned J2EE entity beans (they just made no sense - why would anyone want to try to do so-called object persistence that way?). When I needed some state to span my high-availability, load-balanced cluster, I used a distributed JCache implementation. More lately I've incorporated the use of the Spring Framework for enabling test-driven development of my MDBs and I also use AspectJ for certain purposes.

The Secret Sauce

It is interesting that during a period of time when J2EE was gradually falling into disrepute I was enjoying great success using it as a most valuable fundamental substrate. Yet I was beginning to discern that my methodology of how to do enterprise J2EE-based distributed applications was instrumental in explaining the dichotomy.

My approach was easy to understand, easy to design with, relatively easy to implement, worked robustly when deployed, and remained easy to evolve over time as SOA was an inherent characteristic. I was therefore well pleased with J2EE while others were being frustrated with the complexity of their approach - which was all beginning to look like anti-patterns. J2EE was certainly an uber spec, but it just wasn't panning out in practice for a lot of folks.

Dueling Distributed Communication Techniques Over Lunch

Rather recently I was invited to lunch with some engineers that work for a prominent software vendor. The topic, ostensibly, was to be enterprise application servers. As preface to this meeting I wrote a whiter paper where I set the stage with a short discussion of distributed architecture philosophy. Knowing that what I was going to say would be somewhat controversial, I opted to not mince any words:

"Let me get right to it by lobbing some grenades: I recognize two arch evils in the software universe – synchronous RPC and remoted interfaces of distributed objects."

and

"The network is neither deterministic nor reliable - so get over it."

as well as

"Alas for distributed computing systems there’s no possibility of applying a magic bullet solution that waves away the problem of interface evolution management if remoted interfaces are pervasively how things are built."

and then especially this:

"Seemingly without a clue, though, the major vendors of the computing industry continue to purvey distributed computing communications standards that are by now nth generational attempts at the old saw of the remoted interface. News flash! Details at 11: An interface method invocation that is packaged up as an asynchronous message send underneath the hood does not assuage the intrinsic brittleness problem of remoted interfaces."

Well, sure enough the bulk of lunch-time conversation ended up revolving around this matter of sending data-only messages versus calling distributed object interfaces. I was put on the spot to justify why I'd want to continue with the supposedly "messy business" of writing code that sends and receives messages when I could just call interface methods - which is so naturally intuitive, of course. Yes, I concede that up front calling a remote interface can seem like the simplest approach, but in the bigger picture it has a much higher cost and in multiple ways.

Well, this lunch encounter illustrated to me just how vested the big software venders are in their technology API stacks. Here it is a perfect time to take stock of the lessons learned over the last half decade and make course corrections. That isn't likely to happen, though. The philosophy of architecture I espouse tends to undercut these cool, shiny technology APIs that the vendors' marketing campaigns are busy enticing you developers and architects to buy into. I guess they figure all the money has been wrung out of MOM and these days its become commoditized and open sourced. So is time to herd everyone on to the next great thing that they can lock corporate IT into. Plus the money has already been spent to devise the shiny new technology API stacks and needs to be recouped, or at least justified.

Ted Neward and Effective Enterprise Java

I came into Java enterprise software development in an oblique, unorthodox fashion. Yet imagine my surprise (just a matter of days after the above mentioned lunch meeting) to discover that my personal experiences and architectural conclusions are exactly echoed in Ted Neward's book Effective Enterprise Java (written in the style of the Scott Meyers Effective C++ book - Scott had a hand in helping Ted with his Effective book effort).

I have engaged in email dialog with Ted and sort of know him through that medium, but I've never attended any of his conference sessions or read his books. Basically my exposure to Ted were his articles on C# 3.0 LINQ and a podcast episode on the same subject. Yet while browsing Java books in Barnes and Nobel I decided to pick up this book of his and right off stumbled on to material that...well, I could have penned myself as it coincided precisely with my own architectural rules of thumb.

I became fascinated that Ted and I had arrived at some identical conclusions independently of one another based on our mutual real world experiences of building enterprise software systems. I was doubly fascinated because of this recent lunch meeting experience where it was evident I was coming off as a heretic of sorts to my audience. Naturally I began to take some satisfaction (and relief) in seeing that I wasn't the only one that had come to hold such questionable beliefs.

Excerpts From Ted's Book

Ted's book is written as a collection of 75 items that expound on principles and offer sage advice that are of the sort that tend to come from battle tested experience. I've selected three here that in particular underpin my own enterprise distributed software endeavors:

Item 19: Prefer data-driven communication over behavior-driven communication
(is referring to messaging vs. distributed object remote interface programming)
Three reasons: evolution, intermediaries, and recoverable operations.

Item 20: Avoid waiting for remote service request to respond
Is better to pervasively design network interactions to be asynchronous in nature.

Item 18: Prefer context-complete communications styles
The OOP-centric design approach of remote interfaces tends to be more chatty on the network than sending context-complete document messages.

Of course Ted's book expands on these items for several pages each. Also a number of his other items mutually reinforce these.

A Quick Slide Show That Touches The Salient Points

I recently incorporated the above precepts into a company internal presentation. I've since made a version of my PowerPoint slide show that has corporate-specific material removed:

Data-Driven Communication vs. Behavior-Driven Communication

These principles for building distributed software work very well - that is, they are most effective. From the outset of my uptake of J2EE they have significantly constrained what I consider as worthwhile in an application server.

What is a touch ironic is that Ted aims his book at a Java programming audience that is very likely steeped in J2EE - which tends to imply that they're used to the Session Bean and its paradigm. That is the EJB that exposes a remote interface via the RPC of RMI, CORBA, or web services. The Message Driven Bean was added midlife to J2EE evolution (EJB 2.0 spec). Ted, like many others, has found much to like in JMS and MDBs (otherwise he tends to take J2EE to the woodshed). Even Bruce Tate, before he abandoned Java for Ruby on Rails, wrote positively of JMS and MDBs in the Bitter EJB book.

Yes, Ted's book was published back in 2005 and I'm admittedly rather late to discover it. Yet my recent lunch encounter tends to indicate that the advice it espouses is still as relevant as ever given the agenda trajectories of the big software vendors of our industry. As always, you'll have a basic choice to make: succumb to the vendor marketing noise (likely to be echoed by your upper management and certainly peer pressure), or else think things through for yourself and choose to adopt pragmatic approaches that work very well over the long haul - resulting in successful software systems that are actually pleasing to live with.

Conclusion

Turmoil is perhaps the best word to describe the state of affairs in the Java middle-tier software world. J2EE 1.4 continues on based primarily on existing inertia. JEE 5 is still yet to register an impact - good or bad - in terms of working production systems. Spring Framework uptake continues and in the meantime a revolt arises evidently with the intention of overthrowing all things Java (Ruby et al). Yet in the midst of all this I've been building perfectly fine J2EE distributed software according to my own particular fashion. The secret - ignore the crowd and marketing machine hype - trust in what you know from hard learned lessons. I learned ten years ago that distributed objects don't really work well. The fundamental problems have never been addressed. Happily there exist a much better approach anyway so it doesn't matter.

References

Effective Enterprise Java
Addison-Wesley Professional
Author: Ted Neward

Tim Bray on Rails, REST, XML, Java, and More
Tim Bray, one of the inventors of XML and current Director of Web Technologies for Sun Microsystems, is interviewed while attending Canada on Rails conference. There is a delightful treatment in here of the WS-Death Star stack.

Bitter EJB
Manning Publications
Authors: Bruce Tate, Mike Clark, Bob Lee, Patrick Linskey

Don't Make me Eat the Elephant Again
Author: Bruce Tate

synchronous RPC considered harmful
Yes, yes, a play on Edsger Dijkstra's "goto considered harmful", but of course is right on!
Author: Roger Voss

decoupling request/response
Counter intuitive? Perhaps at first glance
Author: Roger Voss

Building Effective Enterprise Distributed Software Systems
My article is cross-published over on JavaLobby in the Spotlight section. Follow this link if you want to check out the discussion going on over there.

farewell to PowerPC - hello to performance and cool running

2006-08-24T07:31:00.000-07:00

I still wax nostalgic over the Motorola MC68000 of the original Macs. To me it is the best CISC CPU ever devised and a lot of my identification with Apple Macintosh was bonded on its choice of that CPU (which I delved into assembly programming quite frequently).

My first professional job was programming the 68000 Mac where I poped its case and slapped a logic analyzer on the 68000 CPU to assist debugging by trapping on address accesses. Great times indeed!

Then some years later I recall the transition to PowerPC when working at Aldus and getting PageMaker to run on Apple's choice of a new RISC CPU. The instruction set was new and interesting - and code ran faster.

One thing, given the segmented addressing scheme ancestry of x86 and its paucity of registers, pretty much any chip could beat x86 in terms of superior instruction set that a geek would tend to prefer. Any geeks that "enjoyed" x86 at the instruction set level were the masochist - and likely worked at Microsoft. (Though I still managed to write some very worthwhile assembly code on x86 that boosted application performance.)

Yet here we are now with the new Mac Pros on Woodcrest Xeon x86 CPUs. The difference in architecture of this new Mac vs its PowerPC predecessor says a great deal about how far behind the curve PowerPC has fallen. There's simply no contest by any criteria of measurement. Is rather sad that PowerPC innovation - in the areas that count for modern desktop and laptop design - is now out of the race. Even the XBox 360 has a fundamental heating issue due to its PowerPC use (otherwise the XBox 360 is a very lovable piece of hardware from a geek perspective).

At the end of the day I prefer my Macs to be able to stay on the cutting edge of performance and power consumption reduction. It's been years since I've bothered to piddle with programming a CPU at the instruction set level (three, actually). The ugliness and inelegance of the x86 instruction set? I'll just deal with it the same way I do my morning breakfast sausage - enjoy the taste but don't ever bother to visit a sausage factory.

Language Wars Redux - The Imminent Approach of LINQ

2006-05-29T17:45:00.000-07:00

On the plane trip back from JavaOne I sat across the asile from a MS C# compiler developer. We had a lively conversation the entire trip. This ultimately prompted me to look into something called LINQ.

Language Wars Redux

Well, here we are in the Java community having gone through many months of absorbing the new language features in Java 5. We've been challenged and then excited by the phenomena of the dynamic scripting languages (and their frameworks) such as Ruby on Rails and even JavaScript in the context of AJAX. At JavaOne we were given a preview of Dolphin where modifications to the JVM will make it more facile to integrate dynamic scripting languages. A strong case was made for potentially integrating XML as a first class type in the Java language. We were wowed by the Groovy scripting language demonstration and excited by the prospect of the ranks of the Java platform numbers swelling as Project Semplice BASIC offers to give VB6 developers a home running code on the JVM.

These have been some exciting and heady times for the Java language and perhaps more especially the JVM platform. (At JavaOne it was clear that Sun has finally come around to fully embracing the notion of the JVM as a universal platform that should do a good job of accommodating other languages besides Java - the marketing position that Microsoft has held toward the .NET platform from day one.)

Alas, there may be some storm clouds appearing on the horizon.

If you haven't heard of it yet, the day will come when we've all heard of Microsoft's LINQ feature for the .NET platform. .NET Language Integrated Query (LINQ) is more fundamentally groundbreaking in language design than, say, a feature such as generics. And its impact on programmers and how they achieve their end goals will be more dramatic than such phenomena as Hibernate or Ruby on Rails and the revolution of thinking those entailed (ORM/EJB3 db-independent persistence and ActiveRecord dynamic types synthesized on the fly).

There has arisen a mode of thinking in Java land that by embracing dynamic scripting languages we can essentially address shortcomings in Java or bolster Java with exciting new capabilities. The scripting language Groovy is perhaps the ultimate expression to date of this line of thinking. It is an appealing notion - why add new features to Java, such as an intrinsic XML type, when Groovy already has a great markup language feature that makes working in XML very groovy indeed? Attention could instead be focused on making the integration of Groovy more seamless with Java and the JVM - and even making it the flagship scripting language such that it is there out of the box when the JDK/JRE are downloaded (with no additional installation steps required to make Groovy accessible). Given that Groovy scripts can be compiled into .class files it would seem to be the natural vehicle in which to address neat new language capabilities, as it is not limited to just pure scripting such as the likes of a BeanShell approach.

The new challenge: When Microsoft declares C# 3.0 to be production ready, it will have features that essentially leap frog the likes of: EJB3 ORM and its portable query language; many of the features entailed by the dynamic scripting languages - such as the Ruby on Rails ActiveRecord concept; and it will have a very facile ability to work intrinsically with XML. The LINQ feature will essentially unify the representation of query access of data across the domains of relational database, XML DOM, and in-memory collections/object graphs. It will be a language intrinsic capability for universal query. A day will come where it will even be used in every day programming situations - say, query all the fields of a visual form beginning with some name prefix string, and then perform a common operation on all those widgets that the query selected. One of Anders Hejlsberg's favorite demos is illustrating how LINQ can be used to query (and filter with clauses) out of type system reflection information. Thus LINQ is not just for relational database access, nor is it just a replacement for XPath, nor it it just an in-memory database query language. It embraces all of those as it is general purpose - and even facilitates selecting and migrating data between these various domains, i.e., select a data subset from a database while populating it into an XML DOM with trivial effort on the part of the programmer.

The common wisdom preached by the Ruby advocates and exemplified by its neatest accomplishments, such as ActiveRecord, is that such can only be accomplished by the dynamic, loosely typed scripting languages. However, the LINQ feature in C# 3.0 will turn this thinking completely on its head. It will bring in to question whether we really have to throw strong typing away in order to do these neat new things. LINQ creates tuples or anonymous types but it is able to retain type checking. The new feature of implicitly typed local variable declarations make it possible to hold a reference to these tuples in order to access and manipulate them. Unlike EJB3 query language, a LINQ query expression is type checked by the compiler. The end result is the simplicity style of the dynamic scripting languages but with the full rigor of a traditional strongly typed language. Alas, with C# 3.0 the use cases for these dynamic scripting languages just shrank a good deal. We're basically left with just the use cases of BeanShell scripting.≠ (Well, there are the first class regular expression features of Groovy as well.)

The LINQ feature draws on language enhancements that were added in C# 2.0 and of course various new additions in C# 3.0. To fully understand LINQ, start by reading the spec doc for C# 2.0 and then move on to the spec doc for C# 3.0.

In the on-going saga of the Language Wars, Microsoft's new LINQ feature looks poised to kick butt and take names. For in the meantime, over in the Java community, EJB3 persistence and its portable query language syntax will be regarded as the height of Java technology for query. There will still just be XPath for XML, and nothing at all for in-memory object graphs. Tuples? Forget it - will have to bail out to a dynamic scripting language for that kind of thing.

If you came back from JavaOne all pumped up about the state of Java and the JVM as a platform, you've now got yet another anxiety imminently approaching.

A New Java Forms Open Source Project

2005-10-05T08:00:00.000-07:00

If we had a XUL implementation using Java for both the local actions and the remote objects. This should be combined with a good quality visual editor on the client side and a good documentation. There's also a requirement for the renderer itself to provide a good desktop look and feel integration and be very responsive.

Maybe we should look first to existing Java based XUL implementations.

Am in the initial phase of starting a (and what will be an open source) project that is something totally separate from this idea of an AJAX daemon Dashboard.

I wanted a rapid way to generate forms that are implemented in Java. So will take this HTML parser that I use (works very similar to an XML SAX parser) and have it generate equivalent Java Swing code. Here are the high-level features I have in mind:

Support a subset of HTML (things it doesn't actively do anything with, the parser will skip over - HTML is too large of a specification to support handling in its entirety and for my purposes a subset will do fine)
Will typically use the HTML output of visual web page design tools such as Dreamweaver, Golive, or FrontPage (the idea is to leverage such tools for rapid visual form creation and permit the form designer to specify form field validation via such high-level tools)
Generate serialized Java objects as a resource to use to instantiate the form from at runtime (will be a binary resource file in the form's .jar archive)
Translate any JavaScript validation into equivalent calls to Java methods of a Java-implemented validation library (which will be a standard library bundled as part of client application runtime)
HTML events will be emulated via Swing events. Any named JavaScript event handler will be turned into an invocation of a Java-implemented event handler delegate.
All displayable strings will be stripped out into a property resource bundle and use standard techniques to bind to this string resource bundle based on Java localization.
All code that interacts with the form, say, event handlers, etc. will be generated into a separate Java class (or classes). The visual Swing code that is code generated and the form processing code that is custom written will be highly decoupled.
The Spring Rich Client framework project intends to address form data binding so may look to incorporate support of Spring so as to leverage this (as Spring is an open source project).
The means of client app i/o will now be anything that is permissible via J2SE Java class libraries. Will no longer be limited to just XmlHttpRequest. (In my case I would likely choose JMS as I like to build 100% pure SOA middle-tiers consisting of Message-Driven-Beans or MDBs, and usually build rich clients that run in context of an enterprise intranet or WAN. Someone doing something for the Internet might choose web services instead.)
A given form will be packaged up into a .jar archive and delivered to clients from HTTP servers (probably being pulled down by processing logic of a forms engine built on top of Spring). Could also consider folding support for jBPM into such a forms engine on the client side. That's a JBoss JEMS product so it won't be anything intrinsic to this particular project - just something that could optionally folded in.
Webstart would be the means of launching the forms application from the outset.

So that is an outline of what I have in mind so far. No support of XUL because this concept is ultimately a pure Java Swing application. Once one is using Java at all then why bother muddying waters with other native compiled binaries such as the requisite XUL runtime components?

My concept does leverage HTML and one could see that it would be possible to use this approach to turn out both a Java Swing rich client as well as an HTML web app version. If the HTML web app were AJAX, then the very same SOA middle-tier could support both. And there would be no hugely complex and mind numbing MVC framework to code to on the server-side. Consequently overall development would be an order of magnitude easier.

my AJAX web app experience in the early days of DHTML

2005-09-11T18:34:00.000-07:00

Way back in the 2000-2001 time frame I worked for a technology start up in Kirkland, Washington where we were building telematics software systems (computer applications for automotive purposes - navigation, real-time traffic condition updates, email, locality-based marketing, etc.).

I actually pioneered an AJAX approach to devising the user interface for some portions of our embedded application. (Of course at the time I didn't realize it was AJAX as no one had yet invented that marketing buzz term.)

The target device was a prototype AutoPC from Microsoft that ran WinCE, had IE browser control, and XML DOM stuff - very close to their Win32 implementation versions. (Also developed for a prototype device from Yazaki and standard PocketPC handhelds from Casio and Compaq.)

Initially I was trying to create some UI screens using these UI ActiveX controls that Microsoft devised specifically for the AutoPC, but that was all C++/ATL - very tedious, complex, error prone, and going no where fast. This start up company had some superb web developer talent and I thought if I took an approach where I could leverage those folks I'd be on to a more productive, flexible, and better quality result.

The company's flagship embedded application was written in C++ and used GDI for rendering the UI of the visual navigation portions of the app. Also we already were embedding the IE browser control into this app in order to display some traditional web page stuff downloaded from special automotive-specific web servers. So I decided to build out the rest of the UI of this embedded app in DHTML/JavaScript, XML, and XSLT. (In the end I had to use lex and yacc to create my own mini XSLT and also a mini XML DOM system as the versions provided by Microsoft on WinCE had too huge of a memory foot print - a WinCE process has a 32 MB address space limit.)

Our embedded application had a wireless bi-directional communication data pipe to our server farm - Cellular Digital Packet Data (CDPD). Data transfer messages would be conveyed to the device and the UI would need to dynamically incorporate that data into its visual displays and update accordingly. So data was received asynchronously to various web pages being displayed. JavaScript code then accessed the data and updated piecemeal portions of the displayed web page using XSLT and then DHTML.

This all succeeded and the web designer guy turned out great looking UI stuff in HTML while I did all the nuts and bolts to weave it together and make it actually work within the context of the embedded application. Microsoft and Yazaki went on to show it all off at an automotive electronics show in Detroit. Microsoft had a really cool BMW decked out with their AutoPC while Yazaki had a large booth space where they show cased their telematics console running this app. Oh, the UI was all controlled by touch screen on the Yazaki and voice recognition on the AutoPC. Alas capital dried up in the aftermath of the dotcom crash so I moved on.

So in my current position I have had the experience of building distributed applications in a completely different manner. I have built two significant C# .NET rich client applications that interact with a J2EE middle-tier via JMS (Tibco EMS to be specific). I also built a Java Swing utility app where I became introduced to Java Swing layout managers (the constraint behavior mechanisms for .NET forms are not as powerful). These applications are used in the operations of a number of customer facilities - facilities that are directly tied into the life blood of our national economy's daily ebb and flow.

These apps are a definite contrast to the web-based rich client stuff I designed for the Kirkland start up telematics company. One application in particular is a state machine forms application. Its core manager is a form component hosting system that is rather similar to the Spring framework (at the time I had not yet heard of the Spring framework or dependency injection - I was basing the design off of things I'd learned from working with JBoss mbean services). The experience garnered from this architectural approach has been especially informative relative to the prior rich web app experience. A lot of the same things were accomplished but in entirely different ways.

If I were able to opt for a choice between these two fundamental approaches for devising a new app architecture, I'd go with an approach modeled after this .NET rich client forms app - except using Java Swing and WebStart (so would be platform agnostic and have server-based deployment advantages similar to web applications). With a good forms design tool (Matisse?) I could have UIs created that are every bit as nuanced and attractive as what a talented web designer might do with HTML. And with Swing layout managers underlying the form, it could exhibit much superior runtime adaptive characteristics than HTML web pages or .NET forms.

One of the problems with my "AJAX" web-based app is that even though it was a great improvement at the time over raw C++/ATL/ActiveX-controls, it is still a rather tough way to develop. Whereas devising .NET forms ala the VisualStudio forms designer and using .NET multi-threading coupled to Tibco EMS to get the bi-directional asynchronous data exchange is easier and the overall code is much more understandable (very important for code that has to live a long time).

Building solid web apps via DHTML and JavaScript that has asynchronous behaviors and incremental updating of the UI is hard stuff by relative comparison. I still had to resort to writing a C++ ActiveX control to do some true background multi-threading (which was then accessed by JavaScript). Doing rigorous and robust synchronization on a web page is difficult because nothing about a web browser was designed with this manner of behavior and interaction in mind. Everything has to be hacked and what event notification stuff there is in DHTML for doing some synchronization is rather dismal. Also, the entire conglomeration of the various source code artifacts needed to weave such a solution together is not as understandable and maintainable as, say, the C# code of a form assembly (a Java Swing app would be equivalent in that regard).

Creating AJAX rich web apps is tough even when one can assume the precise browser and operating system that will be used. Yet if one is devising for multiple browsers and perhaps multiple platforms, then that opens a whole other can of worms of problems to solve. With a Java Swing approach there is none of that manner of frustration to be confronted with - well, okay, a little here and there but minor compared to the web app dilemma. (I use the MagicDraw UML tool that is written in Java Swing. It works identically on Windows as it does on my Mac. This is a serious app that has graphical interaction that is rather similar to Visio.)

Where the rubber really meets the road for me, though, is the fact that .NET and Java have multi-threading intrinsically designed into them, it works very well, and synchronization of asynchronous behaviors can be done in a bullet proof manner. Plus the code to do it is entirely understandable and can be augmented by incorporating software engineering tools such as UML, etc. One doesn't get into this completely disjoint world the way web apps do where there is HTML design expertise and then some more traditional programming expertise for devising the server-side business logic stuff. With a Java Swing rich client app approach, what a given team programmer knows applies everywhere.

Another important characteristic of .NET and Java rich client apps vs. rich web apps has to do with memory management. The CLR runtime and Java JVM have sophisticated automatic garbage collection with heap compaction. Because the heap is compacted, it is periodically being restored to pristine condition. Long term heap fragmentation is thus not a factor. The .NET rich client forms app referred to above is constantly adding and removing objects on its local cache yet is capable of running indefinitely. That is exactly what its users do - they leave it up and running 24/7.

The HTML DOM and JavaScript memory management system of web browsers is not as sophisticated. I find both FireFox and IE browser are incapable of running indefinitely - eventually their heaps just get too fragmented and they become sluggish. Because JavaScript in such environments is likely to use a simple reference counting approach (ala Perl prior to the 6.0 rewrite effort), there tends to be more problems with orphaned objects (reference counting is more prone to go awry). So with .NET and Java it's very nice to use a programming technology base that makes it trivial to create applications that can run indefinitely. (The objects that need to be deterministically cleaned up are prominent resources in an app design such that one adds the explicit code to deal with that. After several years of developing with .NET and Java I only experienced orphaned objects when was first learning to work with JDBC drivers. I've been well on guard toward deterministic resources ever since.)

What's really sobering to me about this is that the AJAX rich web app development I did way back in 2000-2001 is very simplistic in comparison to the server-side web app development framework stuff that goes on today. With todays frameworks there are tons of moving parts and interacting technology concepts to learn. Compared to the triviality of my .NET rich client distributed app architecture, it all blows me away what the elite web developers have wrought. And yet in the final analysis a rich client app can still be far more sophisticated in its behavior (most of which is non-visible behind the scenes stuff) than even the most gifted AJAX-style web app. Traditional rich client apps may seem like dumb and old fashioned technology to some but the best rich client apps still surpass the best rich web apps in overall user experience. And frankly, the old-fashioned rich client app really is less difficult to build - despite all of today's conventional wisdom that harps that web pages are brain dead easy and everything should be built as a web page. Every once in a while conventional wisdom is just plain screwy.

Yes, the nature of the Internet tends to pretty much compel a thin client approach. Yet if one is building an intranet distributed application, it is worth pausing and considering if the web app approach is necessarily always the best approach.

Richard Stallman and GNU - political extremist

2005-06-30T23:40:00.000-07:00

> you might take a look at the philosophy behind Free Software

Well, I've read the GNU ideological material before and am entirely familiar with what it advocates.

As a property rights advocate and observer of the national historical experience I disagree with practically every point espoused by GNU. Sure, ideas benefit society and in time all ideas eventually become part of the civilization knowledge base. Yet the originators of ideas should have opportunity to profit from their particular advancements. Software is a product that can cost extraordinary amounts to produce (cost of replication is an entirely different and somewhat irrelevant matter). It's no different than any physical product that is conceived, made, and sold to the audience of those that believe they could stand to benefit from said product in some manner.

GNU: "The system of owners of software encourages software owners to produce something - but not what society really needs. And it causes intangible ethical pollution that affects us all."

Actually for those who create commercial software that is not what society really needs, they soon learn of their error by the feedback mechanism of the marketplace. Notice the shear arrogance and conceit of Stallman/GNU as they assert to be wiser than everyone else such that they truly know what it is society really needs as pertaining to the production of software. And the silly appeal to some purposely vague "ethical pollution" - the hallmark reasoning tactic of a conceited group that sets out to assert its moral superiority over everyone else via innuendo.

GNU: "The economic argument for owners is erroneous"

I have no problem with folk that want to write software and give it away for free and share it with a community. People should certainly enjoy the freedom to do as they want in that regard. However to GNU, the so-called "freedom" that they espouse is really only in the fascist terms that they envision it. The true believer fanatics of the GNU creed would prefer that all software be created under the terms and conditions that they advocate. The continuing of existence of a commercial software marketplace of the traditional manner disturbs them deeply - it practically drives Stallman into conniptions.

GNU: "Society also needs freedom. When a program has an owner, the users lose freedom to control part of their own lives."

This is the standard line of all political socialism. The practical problem with socialism, of course, is that it is entirely too one-sided in terms of who has rights. True civil society is founded on a means to arrive at balanced relations between societal members. Hence users of software shouldn't be absolute in their rights as the GNU political ideology would have, but instead there needs to be a balance between those who create/fashion/fabricate and those that benefit by its acquisition/use. Private property rights and the marketplace is the means that societies achieve civil balance between differing parities and thus promote the greatest welfare.

GNU: "It also states that the purpose of copyright is to promote progress---not to reward authors. Copyright does reward authors somewhat, and publishers more, but that is intended as a means of modifying their behavior."

Well, here we have at least a cryptic (and grudging) admission that profit motive was what was considered by the Founders as the best means to promote human progress. The Founders were realist and thus sought how to best fashion government and its rules to the nature of man. The Founders were very wise by and large. Stallman, though, is nothing but a political extremist and his politics is counter to the bulwark of what the Founders fashioned.

Linux GUI vs Mac OS X GUI

2005-06-12T15:32:00.000-07:00

Is true - lots of Java developers indeed look favorably toward the Mac and its OS X:

Are many switching Java development from Windows to OS X?

But there's an even more potent and contentious schism brewing in the computing universe - lots of folks that have been fairly bullish for Linux are defecting to Mac OS X. We still see Linux as quite good as a server OS (the 2.6 kernel multi-threading enhancement finally enables Linux to be a good J2EE app server OS), but we've become completely disillusioned with Linux as ever becoming a viable desktop OS worthy of standing toe to toe with Windows.

There's simply no disputing that Mac OS X is clearly the superior GUI OS which has some Unix-inspired heritage underneath the hood. The Linux GUIs are not even in the same league.

A lot of the Linux stalwart devotees aren't too keen to hear this kind of talk, and naturally some of them get angry. These folks tend to be the free software political ideologues that view the use of computers and computer software as a political statement.

The rest of us folk just want a great computing experience that we can enjoy and benefit by.

The story of Java developers liking Mac OS X is a bit old by now - while Linux zealots vs Mac OS X is just really getting heated up in earnest. (Frankly the Linux zealots are stewing from a bad case of jealousy and envy. The fact that a proprietary GUI OS is so vastly superior to any open-software/free-software GUI attempts is a reality they can't cope with very gracefully.)

The success of Mac OS X is a jab in the eye of the software-must-be-free ideological fascist. That is perhaps one of the very best sociological benefits of the burgeoning success of proprietary Macintosh and its OS X.

ASP.NET vs J2EE

2005-04-30T15:00:00.000-07:00

About two years ago I attempted to use ASP.NET for a project, but after that project was concluded - and due to experiences encountered - I abandoned using ASP.NET and have been using J2EE for middle-tier development ever since.

Here is a run down of some of the troublesome issues I ran into:

I didn't like having to use IIS nor the fact that ASP.NET runs in its own separate process - security models and process/thread permissions get more complicated to configure. IIS has its own security settings, user account and process permissions, and then ASP.NET runs in a separate process with its own user account as well. Because my corporate network environment is very locked down in terms of user permissions, this caused me a never ceasing source of grief and difficulty to sleuth out these headaches.
The normal posture at our company is to not let individuals run IIS on their computer because of it being a target for security attacks. I had to get special domain permissions set up so that I could run IIS on my own computer for development purposes. This did not thereafter make matters trouble free for me, though. I continued to relentlessly struggle against IIS vs ASP.NET permissions configuration headaches, and when I threw in an Oracle OLEDB driver that needed to be used, I got pummeled with yet another very thorny permissions problem that took several days to sleuth.

After all that, all I could muster ASP.NET to do is to be enabled so as to program some code that responds to HTTP request and/or implements web service calls. Well, what I really wanted, though, is to be able to implement a distributed application that uses messaging. However, with Microsoft, that involves licensing and setting up an MSMQ messaging server. MSMQ is a Microsoft proprietary legacy messaging service which was built by their dev group in Israel. (Microsoft does a very poor job of supporting MSMQ with respect to non-Windows platforms - basically leaving one to resort to a scattering of third party products where yet still more licensing cost would have to be incurred.) Because it is a legacy system, to use it from .NET implemented managed code means working through a layer of .NET wrapper classes that Microsoft provides. To top it off, ASP.NET has no intrinsic container-like architecture for hosting components that participate in messaging.

Now .NET Remoting would be an alternative to ASP.NET web services. But unless you roll a standalone .NET Remoting server application (with all of the reinventing of the wheel that that entails), then you still get the convoluted solution pathway that involves the IIS process coupled to the ASP.NET process.

Neither .NET web services or .NET Remoting are messaging style solutions but are just another dressed up RPC (with tons of marketing hype behind them - particularly web services). In the Microsoft camp, one will have to await project Indigo to deliver a managed-code implementation that has messaging capabilities. Now these RPC-style protocols have significant draw backs when being used to build distributed applications: There is high coupling between end-points (they have to know about each other directly); there is no queued behavior of the data traffic; nor a publish and subscribe feature; nor a server-side push notification capability (although with a persistent .NET Remoting connection one might concoct something - but would have to explicitly implement the fan-out of one publisher publishing to potentially many subscribers).

Microsoft does provide the ability for both .NET Remoting and web services clients to do asynchronous calls, but there is no queuing on the server-side so you can't count on a coherent ordering for the processing of request. A client's given async call could end up getting processed ahead of another in a manner that is chronologically out of order. Also, with async-style RPC there's no equivalent to message persistence - which when message persistence is coupled with transaction semantics, it enables messages to be reliably processed (think credit card transactions, etc).

Obviously with .NET Remoting or .NET web services there is no such thing as being able to bridge message traffic or route message traffic based on things such as message properties. One can write filters that can peek at messages before the normal code sees the payload, but these are crude to make use of relative to, say, the Java JMS message filtering approach. Yet with true messaging solutions, message traffic, selectively filtered, can be asynchronously bridged to other queues and thus other message consumers for, say, diagnostic or archiving purposes. (I've used this technique to test software that is under development against actual real-world production message flows - because of message bridging capabilities I could do so transparently in conjunction to the production environment. My program under development could process in parallel to the actual production system in a phantom-like manner. This kind of capability is extraordinarily useful. RPC is stone age computing technology - messaging is 21st century.)

In the particular project where I attempted to use ASP.NET, my clients were wireless WinCE devices running .NET Compact Framework. Consequently I was stuck with having to use .NET web services. Because that is implemented as an HTTP-based protocol - where the client connects, does an operation, closes the connection, and goes away - I had no means of server-push notification or publish/subscribe dissemination of events. (One could resort to UDP broadcast of events but then will get into subnet concerns, etc. That will be yet more hacking and reinventing of the wheel for what should be an off-the-shelf capability. Another technique is to open a persistent database connection and wait on a table trigger event. Such heavy weight connections are problematic for wireless client apps running on WinCE.)

J2EE to the rescue:

J2EE, on the other hand, has JMS as an intrinsic messaging API - which must be supported by a compliant J2EE app server - and the EJB 2.x spec has the EJB variant called the message-driven-bean (MDB). Consequently J2EE app servers come out of the box with intrinsic facilities for building distributed applications on a foundation of true messaging.

Unlike the shotgun wedding of the IIS process coupled to ASP.NET process, everything that I program to in the J2EE spec runs within the same operating system process. (In actuality I happen to use a Tibco JMS messaging solution which runs in its own process - but the integration of Tibco into my J2EE app server is such that this fact is completely transparent to my Java EJB code.) Consequently I never have permissions configuration headaches the way I chronically had to deal with in the .NET/Windows universe. The security and permissions model is vastly easier to contend with.

For my Java development I routinely have used solutions such as Hibernate ORM persistence. (Only recently has Hibernate been ported to .NET.) Instead of data transfer objects coupled with entity beans, I just use plain java objects to fulfill both roles as ORM persistence easily facilitates doing such. In ASP.NET there is the ADO.NET data set. Not exactly object oriented to where your OOP-centric object graphs and entities can be one and the same. Nor are ADO.NET data sets truly suitable as data transfer objects (the objects I shove around via messaging). Its not good to so tightly couple client code to entity structure. (Microsoft has for years now foisted such undesirable tight coupling, though, due to their fundamental approach to data persistence - which they show now sign of relenting from.)

I routinely use XA where my EJBs are doing messaging and also interacting with a JDBC resource (either directly or indirectly via an ORM layer). For J2EE EJB this is done with simple declarative statements in the XML config files of the EJBs. With .NET I'd have no XA support for doing messaging and database persistence coupled in the same transaction because there's no comprehensive app server architecture for hosting such. Instead of relying on ASP.NET to provide XA capabilities via an integrated distributed transaction manager, one has to resort to COM+ MTS - yet another Microsoft legacy system that has not been re-implemented as managed code. (MSMQ, COM+ MTS, and the lack of a managed code ORM persistence service, are all visceral illustrations of how .NET was conceptually delivered into the world as still born.)

My distributed apps frequently employ JMX mbeans to do hardware device control - where they are hosted in the J2EE app server. By being hosted in the app server they can readily provide service interactions to my EJBs and also avail themselves of all the various standard J2EE capabilities. Again, ASP.NET has no provision for hosting such open-ended components - one would have to run them in a separate, dedicated server process (that one rolls oneself) where communication with other ASP.NET-hosted components would all be inter-process in character. Oh yeah - JMX mbeans make it easy to add management interfaces to applications hosted in J2EE - which is of course their intended purpose. There is nothing comparable to the JMX mbean for ASP.NET. The beans I write for J2EE are split about 50-50 between EJBs and JMX mbeans. The JMX mbean is a very useful adjunct to EJB.

In my last significant project I used a rather nice distributed cache solution that is available for my J2EE app server. It likewise supports XA. This cache turned out to be an ideal solution to a issue that this project had to address. My future projects will be leaning more and more on making use of a distributed cache. The advent of 64-bit computing and JVMs that run in 64-bit mode is going to open the door to doing some radically new things with distributed caching. There is nothing out of the box to do something comparable with ASP.NET.

In the future we'll also be looking to adopt a BPM engine - there are scads of them to choose from that can be dropped into a J2EE app server.

My problem domain is such that I don't have to worry so much with scaling of the middle-tier, but I do have to support high availability. Hence for production my J2EE app server is indeed running in a cluster. All my applications that are deployed are designed to run in a cluster configuration (hence the value of such things as a distributed cache). My J2EE app server makes building cluster-capable applications a simple matter. Even my device control JMX mbeans support high availability where they are automatically managed to run as a singleton in the cluster. With ASP.NET one has to resort to the Windows OS to provide some capabilities to enable running web services and such in a cluster. (In contrast, my Java-facilitated clustering capabilities work regardless of whether I'm using Windows or Linux/Unix for the operating system.) There is no .NET distributed cache service per se - one can buy third party products for this, but be careful to nail down the matter of whether such a product can support XA under COM+ MTS - otherwise, what's the point?

So do you see the repeating pattern with Microsoft? You have to bring in: their MSMQ server product for messaging (and be left with very poor heterogeneous platform support options); their IIS web server for front-ending anything using HTTP/HTTPS; COM+ MTS for XA management; Windows 2003 Server for clustering features; ASP.NET for running components written in managed code, and resort to some third party product if you want a distributed cache. In the Java universe we can easily get all we need in a single J2EE application server product - with perhaps a service or two provided from elsewhere such as external JMS or JNDI/LDAP.

Now in recent months my product support organization has made the decision to move our middle-tier applications to be hosted on Linux servers. It has been a breeze to move my Java J2EE stuff over from Windows to a Linux platform. Obviously this would not have been a viable option with a .NET middle-tier approach. Maybe some companies are okay with IT decisions that lead to intentional platform lock-in - myself, I regard such as irresponsible and essentially being cavalier with stock holder investment. I suppose I see that as the biggest negative of all regarding .NET technologies. Unlike Sun, Microsoft has made no earnest effort to make .NET truly OS platform agnostic (you don't see Microsoft themselves actively porting .NET to Linux, Unix, or Mac OS X - yet Java runs on those and even on AS/400).

IBM's AS/400 server rocks the house in two Java benchmarks

As one can see, real world distributed software systems cannot be built without availing oneself of all manner of Microsoft Windows-specific legacy systems. And in going down that path there will be many compromises, frustrations, and outright integration problems to overcome. Alternatively, in the Java universe Java is effectively its own platform rendering the underlying OS secondary. So with a single J2EE app server acquisition one can get just about all that you need. Then there's the various Java IDEs and open source development tools that provide a rich development-side infrastructure.

Jettisoning ASP.NET while adopting J2EE for middle-tier development is one of the best decisions I've made in my career from both a business point of view and a software developer's perspective.

Use AOP to wrap objects(and solve thorny dependency injection issues)

2005-03-19T08:31:00.000-08:00

At first glance Java was supposed to have this garbage collection feature that would cure the common cold and world hunger.

It was soon realized, though, that there are kinds of resources other than memory that require deterministic cleanup. An example is a database which has a license that restricts maximum number of connections. Any Java code that uses db connections needs to deterministically release a db connection as soon as possible in order to maximize the utilization efficiency of this limited resource.

Because this is a reoccurring problem, Java classes that encapsulate such resources tend to provide a close() method, etc., which can be explicitly invoked to deterministically release the underlying resource. (The C# language introduced the IDispose interface and 'using' keyword as features in that language to effect this particular pattern - and works fine for the occasional case where the resource is only used for the lifetime of some local scope of activation.)

Well, relatively new concepts, such as dependency injection (where a framework is in charge of setting references to dependent collaborating objects on behalf of client code that it manages), introduce yet an additional strain on Java programming practice.

If a framework is in charge of providing and setting references to dependent resources, then it will need to be in charge of managing the continuity of these dependent objects. The client code that is managed by the framework should never have to worry with the lifetime availability of the collaborating objects it makes use of or else the whole notion of dependency injection breaks down. (The goal is for client code managed by the framework to solely concentrate on business logic and not unrelated concerns.)

Opinion: Injection in EJB going too far?

So the framework provider could provide a wrapper class for every such dependent object that it has responsibility for injecting. This wrapper code would intercept all method calls such that when invoked there will be an opportunity to reestablish the availability or viability of the wrapped resource (in the event it has gone away or gone bad), and then complete the call transparently to the caller. Effectively there is a proxy object that is intercepting all access to the underlying wrapped object. It is a reference to the proxy that the framework injects into the client code that it manages. The proxy is then responsible for managing its underlying wrapped object.

Implementing such wrappers for complex classes would get tedious very quickly, though. What is needed is a language feature that simplifies wrapping other classes with such pervasive interceptor mechanisms. Perhaps what is needed is something analogous to the C++ ability to create smart pointers via overloading the pointer access operator. Well, one could indeed add a new Java language feature - but in contemporary Java the thing to do would be to use AOP to byte code enhance the public methods and properties of the target class. One would have to specify the pointcut in xml AOP language instead of rely on an AOP annotation because most typically the target classes one wants to wrap are being provided from third parties in the form of .jar libraries (i.e., a database JDBC driver) where there is no access to source code.

The upshot is that this problem is quite solvable and with the advent of AOP it should be straightforward for framework providers to implement. Most frameworks which are providing the feature of dependency injection are now also general purpose AOP frameworks. Hence the tools are already present. Contrary to the linked discussion above, the is no inherent fatal shortcoming to the feature of framework provided dependency injection. The problem issues that were raised are entirely solvable with application of yet more AOP.

current state of Microsoft .NET

2005-03-09T00:14:00.000-08:00

[my rebuttal]
One assessment I agree with Mr. Grimes on is what a total waste of time VB.NET was. He was a little harsh on the size of the .NET Framework - for enterprise intranet rich client development it's no big deal to install 25 MB on the client workstation. I know one thing for sure - I'd much rather develop our rich clients using C# .NET rather than C++ and MFC/ATL.

He's also correct that WinFS was another MS misstep - just like the Object File System a decade before it. Google local search coupled to conventional file systems will rule. Turning the file system into one big SQL server storage system was a bad idea - chiefly because of the burden it places on developers to devise storage format concepts that accommodate this approach (and where's the ORM solution that eases all the pain?). And just how normalized will that storage be (and secure in an isolated sense)? Where's the crisp distinction of data borders that we have today with files?

The real problem with LongHorn is that it's so far behind the curve. On Mac OS X I'll soon be enjoying the Tiger release. Given the Mac's already existing easy-to-use advanced UI (with cool 3D effects) and the new Tiger features, LongHorn will be long preempted elsewhere before it ever sees the light of day in a shipping version. The next couple of years will be a wonderful time to jump ship from Windows to Mac - can you say "Mac Mini". (Now I can relegate my Windows boxes to being headless servers - the same way I use my Linux boxes. That way they can't cause me near so much pain and sorrow.)

And then on the server-side - well ASP.NET is pretty much a one trick pony. Not a very rich server-side development environment to work with there, alas. The puke of .NET Remoting or Web Services over HTTP - pick your poison. Yeah Indigo might finally provide a managed messaging implementation (just like ORM this is another feature of .NET that was axed back in '99/00 timeframe when I was still there), but Microsoft's idea of heterogeneous messaging is “between different versions of Windows”. The prominent JMS implementations will be over a half decade old in maturity by the time Indigo is shipping code. The leading JMS vendors are already heterogeneous in both programming language and platform that are supported for messaging.

Indigo as described would indeed address some of what is retrograde about ASP.NET and Web Services combo as they exist today (except that there’s still the issue of addressing transparent clustering for scaling and high availability - as well as replicating transactional distributed caching that will enable capitalizing on 64-bit address space and multi-gigabyte RAM in current generation servers). It is just that with the years of maturity of the enterprise Java platform, where all these capabilities already exist in spades, the question any responsible IT staff will ask is why put all of one’s eggs in one “Microsoft basket” - and for a set of technologies that are a year or much longer from being shipping code? Microsoft historically has done well with the mid-tier and lower-tier IT crowd of VB developers, but for really serious enterprise IT development needs they've always been a day late and a dollar short.

there are no good data entry forms systems

2005-01-16T01:14:00.000-08:00

A few months ago I was fed up with the technologies available for data entry forms as I surveyed the forms landscape. Of the various choices of how to implement forms applications, they all suffer drawbacks of one sort or another.

First of all let me state my bias: I currently build distributed applications that are used on internal enterprise LAN/WANs. I don't build systems that are accessed over the Internet by arbitrary external customers (I've also done that kind of stuff before too but in a manner completely different than what everybody else is doing. More on that later.)

The systems I currently build are J2EE for middle-tier and C# .NET rich client apps. I use Tibco EMS for JMS messaging to weave everything together - I stay away from RPC and HTTP as much as possible. If I need to stream a megabyte or so between nodes then might use HTTP to do it - otherwise stick with JMS messaging. It's essential to have an event driven distributed application architecture for my problem domain. I use XML currently for message formats, described in XSD. JAXB is used on the Java side and XsdObjGen is used on the .NET side.

This all works fine for my rich client apps. These client apps are not form intensive themselves - they tend to host systems that are, though, and also do things like interact with PBX and streaming video.

But I'm now considering how to address the problem of a forms hosting system itself - one that may need to push 200 to 300 potentially different forms out to end-users for various data processing and report generation. (The current form application technology my company uses is being retired by the vendor and so we need to switch to something current and more or less mainstream.)

So some folks say we should adopt HTML forms and one of the various MVC solutions for managing the presentation layer (our business logic will stay encapsulated where it is so we're already by default looking for an MVC approach).

We'd like to have the business analyst use high level GUI tools to define forms, data validation, and perhaps business rules. Hopefully the forms presentation code could thereby get generated without much need for procedural coding of the visual rendering aspect of the form. Hopefully such tools spit out mostly declarative descriptions of both appearance and data entry validation rules. There's also a consideration to perhaps tie our new system in with BPM. Some BPM systems have their own GUI business analyst tools. Some of the open source BPM tools just use text file descriptions created in a text editor.

Now as to HTML forms and HTTP protocol - I have very definite issues with that mix of technology.

With HTML the presentation is page based (ala the HTTP protocol). But I may want to update new entries into a UI widget by pushing the updates directly to the relevant widget. I don't want users to have to suffer entire page refreshes of a form just because some of the data of the form they're working with is being updated on an event notification basis from elsewhere in the distributed system. Okay, so DHTML can be used to graphically modify an HTML form widget, but there's still the issue of how the update is pushed to the widget. (The updates need to be pushed from the middle-tier side to the clients. Naturally JMS topics work great for this.)

I don't care for the flat list of scalar values that HTML forms use to return submitted form data. I'd prefer to submit the data as a structured XML document (so can be unmarshaled into an object graph at the middle-tier - where may use Hibernate to persist, etc.).

Sometimes there will be fields (or widgets) on a form to where an action or event must be fired across the distributed application based on a user interaction with the field. Such actions will not entail submitting the form data for processing. (Again, JMS messaging is more suitable than HTTP here for firing such discreet events.)

Other things I don't like about HTML is having to use JavaScript and a single threaded programming model for the form's client-side runtime execution. The rich client forms I build in .NET constantly make use of multi-threading and background JMS message processing. Yet if I start incorporating a Java applet with the form in order to get decent multi-threading and JMS messaging, then just why am I bothering to use HTML/JavaScript and HTTP to begin with?

I don't like the synchronous nature of HTTP protocol. If I submit data from an HTML form for processing, as a user I have wait for the middle-tier to complete processing and return a result on the HTTP PUT or GET operation of the data submittal. It's very silly to tie up a client in a mode waiting for that result to come back from a busy server. In contrast, my rich clients are able to submit data as an asynchronous JMS message which is placed into a queue. My rich client apps are not modally locked into the expectation of a synchronous return result. Instead any result from middle-tier processing can simply be returned asynchronously on an event notification basis.

Then there's the issue of transparency of failover for when a middle-tier cluster node fails. Tibco messaging enables my clients to transparently fail to a backup JMS server. It's all dealt with by software in the Tibco JMS implementation. Achieving transparent failover for HTTP-based web clients requires more elaborate techniques. Because JMS is highly decoupled by nature, the clients are unaware if a J2EE processing node goes down. Tibco takes care of load balancing to the J2EE cluster nodes. If a node goes down, the survivors simply process yet more messages. So I ask myself why invite the headaches of HTML forms and their HTTP protocol when Tibco EMS solves transparent failover quite nicely and easily?

In the past I've dealt with all these myriad shortcomings of HTML forms by coupling them with a Java applet that does messaging to a proxy server that in turn does HTTP to a web server. In that particular case I was using the messaging system to mirror form interactions to other users that were co-browsing in a common browse session. The interactions of one user are seen by all other participants in the co-browse session. The specialized co-browse proxy server and the web server both exist at the middle-tier and are in a common cluster.

I still need something, though, that is like XForms to where it is possible to return form data as a structured XML document. I don't want to be limited to having to use an XForms compatible runtime because I haven't liked any of the XForms capable systems that I've looked at so far. It's all experimental. There is yet nothing mainstream in XForms. And there will definitely need to be GUI tools for this XForms stuff - a person wouldn't want to try and code all that verbose XForms XML gibberish in a text editor. It's only possible to comprehend the simplest forms that are expressed using XForms (coupled to XHTML or SVG for presentation).

And no matter what manner of declarative form language I wind up with, I'll likely have to use Java applet code coupled to DHTML to do discreet operations on specific form widgets. The Java applet code interacts with the messaging system and then interacts with appropriate DHTML widget. Yeah, I'll probably have to hack the vendor's form technology in order to accomplish this in the way I need it to work.

You know, when it comes to forms technology, I feel like I live in the dark ages. Nobody has come up with a decent system for enterprise forms development purposes. The vendors should just take their HTML/HTTP forms garbage and cram it up their browsers. It's like they all imagine that we're all out there just creating Internet sites and the lame limitations of HTTP pull technology that go along with that gig.

Okay, somebody will suggest using Citrix Metaframe. I've ran these kind of systems. It works great in a lot of respects. I actually love Metaframe as a product and have great admiration for it. It can open new doors for the kind of commerce that can be done over the Internet. A client app form technology implementation can be completely arbitrary using Metaframe - so long as the forms engine runs on Windows. You certainly get the superb capability of a supervisor being able to shadow in on the form entry session of an employee. None-the-less, Citrix Metaframe still doesn't address what to use for the form technology itself.

Microsoft Titanic, Meet Mr Open Source Iceberg

2004-10-03T08:26:00.000-07:00

Summary (of linked article)
So it's begun. Microsoft is admitting in its financial filings that Linux and open source are eating into its revenues. Worse for them, they say, is that what they're seeing is the tip of the iceberg.

[my comments]
Microsoft could start from scratch to build a new OS that doesn't have all the problems found in the Windows architecture. Otherwise Linux will just keep getting better and MS will just be slapping band aides on Windows in the meantime.

For a completely new OS they could adopt the mySQL dual licensing model from the very outset and call it "open source" so that they'd be in step with the ideological politics of the times. (mySQL certainly seems to have the smartest business revenue model of any of the open source products.)

A new kernel at the lowest level could be designed with concepts that promote security and rock stable software. The .NET abstraction could be rolled on top of it (like Longhorn is doing). However, it would have no ability to run old Windows software - just new managed code software. (Any manner of support for the old software is nothing but a security and stability quagmire - which is the foundational problem of Windows to begin with and is primarily why Windows is loosing business in the server market place.)

Obviously it would be primarily a 64-bit friendly OS from the outset but could be scaled down to 32-bit for the embedded marketplace. (By the time this OS would surface in the marketplace, 64-bit would be predominant in desktop/laptop machines, in addition to servers. So 32-bit will be cell phones and control processors.)

The enterprise flavor would be the first natively cluster-aware OS so that the very OS layer itself would make simple the activity of writing enterprise applications that can scale and be highly available in a grid/cluster configuration. Hence this new OS, in its enterprise flaover, would incorporate all the features found in something like JBoss: hands-off transparent clustering, transactional distributed caching with replication (JBossCache), transparent ORM persistence (Hibernate), easy declarative distributed transactions, etc. Messaging would be the substrate for all communications. Even RPC, where provided, would be layered on top of messaging. Effectively it would be a fully grid-ready OS in its enterprise incarnation.

The whole Longhorn thing was such a major disappointment for its lack of imagination and vision. It didn't go near far enough in addressing OS fundamentals, but concentrated almost solely on end-user experience stuff. That has been Microsoft's problem all along from the very beginning with Windows. They deal mostly in eye candy while practically ignoring the termite eaten wood underneath.

Linux itself is based on OS ideas that are 30 years or more old. To write modern enterprise software Linux has to have Java slapped on it. The ancient notion of restricted user-mode vs. privileged kernel-mode is not an adequate concept for promoting security and stability. Native instruction code execution is too dangerous to permit in user-written software. A modern OS design should permit only managed code execution of any user-written software - including device drivers (special memory management features could be provided to deal with heap garbage collection vs safe interrupt handling).

Microsoft could out-do Linux in coming up with a modern OS. They have the money in the bank to fund reinventing themselves. The question is do they have the imagination and the spunk? They're known for being fierce competitors, but are they really? Longhorn was not a very "fierce" response at all to their predicament. Microsoft needs to start by firing a lot of VPs and axing a lot of looser projects. They need to trim themselves back down to a lean machine. They need to cease out of control sprawl that is expansion for the sake of expansion. They need to make it known again that they're first and foremost a software development company - and one of the very best at it. They need to make it clear to the marketplace that they're going to deal once and for all with being known for having a shoddy OS that's dangerous to business. Stock holders are going to start getting restless with Microsoft's listlessness if they fail to demonstrate bold action at this crucial juncture of company history.