How did we do? Well, it has been a mixed bag. Our interest in keeping the solution cross platform quickly ran aground as we faced the difficulty in getting the more entrenched RPG programmers to embrace Java as well as finding Java programmers who were comfortable in the System i world. We also got some push back from customers who also did not have the Java programming resources to grow and enhance the solution. We also began to use, through the hiring of an individual, an ILE RPG CGI framework that would be more “comfortable” for the RPG programmers to use and be more productive. This two pronged approach was completely wrong for a host of reasons and has hampered, IMHO, the overall solution. The business model we used, using support revenue to fund the development of an Open Source solution is actually a good one and has been successful overall. Had those resources been more carefully managed, matching development costs more closely to the revenue model, it would have been an excellent way to grow the overall effort.
Focusing on the development approach: We completely underestimated the effort required to build a web application. I don’t care whether it is written in Java or ILE RPG, the skill set needed to develop a UI using HTML, CSS and Javascript plus backend business logic and framework components completely overwhelmed our programming staff. It was recommended very early on that we hire at least two “HTML Monkeys” (as one programmer called them) to offload the UI to folks with the proper orientation to UI. That recommendation was never followed but would have saved countless hours of programmer time. I honestly don’t think a programmer should be designing a UI in a web solution. Period. I also don’t see any more productivity in either ILE RPG or Java (the only two programming languages we used). If the programmers could have focused on writing business logic and enhancing the framework, rather than designing UI, we would have been much better off.
If I was starting out today with the same code base and the same goals, I would do the following:
- Use HATS to get a web interface in front of the customer NOW. I would do this because the lengthy wait for a completed solution led to some attrition in the customer base. Better to have a transitional technology in front of the customer now to hold their attention rather than to try to hold them with promises. HATS was not mature enough in 2002 and the System i not yet powerful enough to make HATS a viable solution at the time. It is ready for prime time now.
- Decide on a single programming language and go with it. Personally, I like the idea of Java on System i because it makes a solution potentially portable to other platforms while leading with the strengths of System i. However, ILE RPG is a “natural” for the System i and many folks who are already on i are more comfortable writing RPG code. Naturally, RPG will be more productive for RPG programmers and Java more productive for Java programmers (duh!). But there is nothing inherently more productive about either language (although the OO nature of Java might lead to more productivity over time, IMHO).
- Find a framework that is productive. Tough call here and this is where the “missing link” comes in, particularly for RPG programmers. IBM provides no tools or frameworks in RPG that made the move to the web easier. CGIDEV2 was a community response to that deficit,and a good one, but this deficit in in IBM supplied RPG web frameworks still remains. When I look at other tool providers, the successful ones who had proprietary languages that did text based applications migrated their tools to the Windows UI and then on to the web. Microsoft in particular did a great job with this, moving Basic to Visual Basic to VB.NET. Yes, very different languages but MS at least developed the solution (or bought it) and got it in front of their developers. IBM got with the Windows GUI program with Visual Age RPG but then when the web appears they did … ? I am mystified that IBM who fully “owns” RPG and clearly saw the web as the next UI wave simply froze and did nothing to enhance the language to easily get applications to the web. What were they thinking?
We have an awesome platform, loyal customers, and a cadre of programmers just dying to cut their teeth on an IBM supplied, leading edge, ILE RPG based web framework. Where is that web framework, now 12 years after the popular advent of the Internet? That is the missing link. A “link” sorely needed as the platform struggles to maintain market share.
There are many, many other lessons learned from this experience and I am certainly ready and will to share them. Feel free to chime in.
You hit the nail on the head – “If the programmers could have focused on writing business logic and enhancing the framework, rather than designing UI, we would have been much better off”. I worked on a project that did just that.
I worked with an experienced agile architect last year and he used Peter Coad’s Feature Driven Development (FDD) technique to lead the project members in focusing on the business logic. For two months we developed business logic with minimal UI (just enough to accept input and show the results to the client) and no connection to the database. We later added persistence and more robust UI features including Ajax.
I realized later that the focus on pure business logic kept us from creating screens and tables that we would later have to modify. In that way we avoided project churn. The cost of churn, re-doing whats already been done, can stop a project’s forward momentum completely. We were able to do things right the first time by delaying activities until sufficient knowledge existed to do it right.
My project was new development so FDD worked well in that arena. In your attempt to refactor an existing app leaving business logic in the RPG, I would want to proceed as though I was developing in a pure FDD model, i.e., have the client identify a few important, core features for each iteration and then work to deliver them.
Greg,
Thanks for the comment. I equate programmers getting enmeshed with UI design and endlessly tweaking UI with the elimination of secretarial and administrative help and having managers and staff do their own correspondence and other minutia. Sure, you are capable of doing it, but is it really an efficient use of the resource?
Agile development techniques can be helpful because it can focus you on the essential function and keep you from getting sidetracked. In our case, the business logic was encapsulated by the RPG put the application in RPG III did not have a modular design that allowed use to leverage it. We simply re-wrote the business logic in Java and RPG. However, had we focused only on that, as you illustrated, we would have had a much more complete solution much earlier.
Yes, the FDD technique gave us cohesive business logic that was uncoupled from the UI and the DB, i.e., pure pojo business components. With that in place, we could create new business logic or refactor with little or no impact to the other layers. Being able to refactor allowed us to refactor for reuse opportunities and thus we kept the codebase DRY.
I like RPG but when I see the argument that RPG is as good or better language for business as Java I know the person making the argument does not know Java and OO. Abstraction, polymorphism, encapsulation, etc are all tailor made for business logic. Reuse of RPG components in a web application is an interesting challenge but a Java solution using Spring and Hibernate should only half as much code as an RPG solution.
As one of the persons referenced in Pete’s article and one of the original shareholders of the company reviewed herein the biggest thing that struck me was the amount of history that was left out (nor could fit in an IMHO column).
Pete’s description of the “two pronged approach” underplayed the struggle between those who favored cross-platform development vs. those who favored a System i centric solution running under the native virtual machine, where Pete and I were the strongest voices in the respective opposing and competing camps.
Perhaps predictably, our customer base and the company line of credit were unable to sustain competing development tracks and we eventually splintered. By May 2007, five of the seven members of the original management team had resigned (including Pete) and about half of the developers were laid off (including all the Java developers).
Having all the Java developers gone was ironic because the cross-platform approach was favored by the majority of management, though somewhat out of sync with the customer base. We also had a small .Net based development team working on another product line, but it was spun-off and sold to another company.
Finally it appeared that the company was committed to shifting in whole to a System i centric approach using my ILE frameworks and tools but at the same time the majority shareholder began pressuring me to change the terms of our partnership and license agreements to be more favorable to him, which ultimately forced me to withdraw from the company too.
Today the company has substantial software assets, but most of the original developers are off doing other things, leaving a small number of people supporting the Web-based product line while the company reassesses it’s development options. The last word I heard was that they were going with PHP, while I continue along the ILE track.
Nathan is correct. Much *was* left out. All of it documented in very long emails between myself and members of the management team (including Nathan). I didn’t include it because of length and personalities. Including that information didn’t really enhance the article (might have provided some ’emotional tension’ though).
Something to note, however, and I think this is important: You *really* need to have a development plan and then follow it. In this case, we started with a cross platform approach and then added an ILE, System i centric framework (a good one, IMHO, see: http://www.relational-data.com ) that was basically at cross purposes with the original stated direction of development. This wasn’t a bad thing, it just was confusing to everyone on the original development staff (and it was a reflection of what was to come, management-wise). Had our original stated goal been to develop a System i centric solution rather than a cross-platform solution then no acrimony would have ensued. However, the solution, which resulted in this two pronged approach, was a disaster, for all of the reasons I stated before.
Nathan’s prologue is right on. The company is a very different entity today. And, chasing the “holy framework grail”, they are investigating PHP. The problem with this approach is that the framework isn’t the problem. The management of company is. Had company management stayed out of the control of development and the Java OR the ILE solution been adopted (not both) the company would be in a much better place today.
Another thing learned is that in developing a successful software company, technology isn’t the issue, management usually is.
There were pros and cons to the “two pronged approach”, but I agree with Pete that the cons outweighed the pros, and eventually led to the implosion of the company, and a lamentable loss to the customers who funded it. Nevertheless if we brought the same people together today and tried to pick one or the other technical tracks, I think we wouldn’t be able to agree, and that’s a reflection of IT organizations generally. There are a lot of competing alternatives in IT, and most IT organizations struggle with disparate technologies. Furthermore the two pronged approach served the interests of a couple key people in the company. If you’re interest is control over a company that begins on a level playing field, one technique is use the competing interests under the pretense of the role of mediator, which over time puts you in a position to negotiate concessions from one camp or the other through a position of power. Then foment the use of an offshore development group in India which adds to your control – something of a “divide-and-conquer” while appearing to mediate style of management.
On the pro side, we did more than due diligence in proving both J2EE and ILE development options and gained a lot of knowledge from it. Indeed we spent nearly 5 years testing and proving both. Unfortunately most of that knowledge appears to be lost as the principals were forced pursue different paths as of late.
On the con side, it established an on-going conflict of interest between the J2EE camp and the ILE camp specifically, and other camps generally. Detailed designs about our respective architectures under a different setting might have been protected as trade secrets, but in a single-company setting forced to share with the competition, so to speak.
The ILE team had a jump-start with my Web development framework, while the J2EE team struggled with alternatives, so the J2EE team was playing catch-up from day one and spending a lot of extra time reinventing the wheel, as they say. One of the big problems for the J2EE team was that their program models were patterned after the ILE models (a management decision) so the screens were quite similar, but the performance between the two was about 10 to 1 in ILE’s favor – which the customers noticed, and they also didn’t like the extra work to set up and maintain the J2EE runtime environment (Tomcat or Websphere).
Notwithstanding the success of the ILE prototypes and applications, three Vice Presidents who all favored the J2EE track felt they needed to ride herd over me which was overly constraining.
There was a lot of time spent on politics and positioning that could have been spent on development.
As resources for funding new development became constrained over time there appeared to be less communication and cooperation between the development teams and key people appeared to become more distanced and self-serving.
Finally the company imploded and splintered, leaving most of the original principals pursuing separate paths, while a few key people try to rebuild from what’s left.
“Nevertheless if we brought the same people together today and tried to pick one or the other technical tracks, I think we wouldnt be able to agree, and thats a reflection of IT organizations generally.”
I am not sure I agree with this. Were we restarting today, the availability of less expensive hardware in the System i arena and the need for IBM to find, and perhaps fund, a software solution on i, would lead me to conclude that a “pure” ILE RPG solution could make sense. Certainly the past four years of experience would lead me that direction as well.
The cross platform approach was decided before Nathan was hired and was based on several factors that were debated at length. Had Nathan been part of the mix early on, we may have arrived at a different conclusion and a different strategy. Hard to say at this juncture.
That doesn’t mean that I think a cross-platform approach isn’t valid. It is just that in this case, and because of what we were trying to accomplish, an ILE RPG approach would have been less risky overall and (hindsight is 20/20) a faster way to get where we wanted to be. It might have limited future sales but the lower hardware costs on i today would have ameliorated that. Most importantly, the existing customers on i would have gotten their new code quicker.
I’d definitely do things differently. The tools are better than they were when we reviewed options in 2001 and the System i a very different animal. The mistake would be to do this the *same* all over again. That ain’t gonna happen.