Could not resist myself from posting this ! All manager's are currently in this state.

The increasing demands of the customers is to build sites that scale when the traffic goes up and give superb performance. In the days of facebook launches and twitter invitations and previews, the traffic on the site becomes unpredictable. To be handle loads of traffic coming in, application architects need to exploit features that can help scale up the site.

One of the biggest challenges for a site that are heavily dependent on database, where to cache all that data.

To solve this issue, we have multiple products offerings in the market- Gigaspaces, Infinispan in the open source world. Vendors like IBM and Oracle have re-branded or released products like WebSphere extreme scale , Oracle In-Memory database.

For the discerning customers that are mostly IBM shops, the WebSphere extreme scale is a nice product that works very well with the existing WebSphere Stack of Applications. There are some good resources available on the same

IBM Redbooks: User's guide to WebSphere eXtreme Scale

Another upcoming good book is the IBM WebSphere eXtreme Scale 6 by Anthony Chaves which promises to demystify the data grid concepts and their usage in the applications. I will post my review on the book pretty soon.

Today is the last day in Wipro. After 6 and half years I have decided to move on - to explore new worlds; to seek out new life and new civilizations;

I will co-founding a SAAS based service offering for which I will announce the details soon. Lots of work and learning ahead.

Looking forward for support from all my well wishers.

In my day to day role, I am architecting enterprise business applications using multiple COTS product(s) and creating solutions around them.

Following are the architecture principles; I tend to stick to when architecting solutions

1. KISS (Keep it Simple and Stupid) – First cardinal rule I always follow is to keep the architecture simple. I make sure that I do not use jargons or abbreviations that require googling every 2nd word. I create multiple views of the architecture for different stakeholders (IT, Business, Implementation Team, Infra Team and so on) to convey the design and architecture
   

Google Wave is going to change how we collaborate and work. The communication tool aspires to redefine not only email, but the entire web. You can read everything about Google Wave here and here.

What I found interesting is - what is the key difference it brings in ?

In the current model of working, when individual sends out email to people, the email gets stored in people’s mail boxes (email folders). So everybody gets a copy of the mail. Subsequent replies are also routed to everyone mail box. The email client will try to group the email based on the subject as one conversation. Using the conversation mode, one can go through the mail chain to find out what happened. 

In this model, we usually face 2 main issues

Whuffie is a general measurement of the overall reputation, and Whuffie is lost and gained according to one's favorable or unfavorable actions.

In the enterprise context, Whuffie is also an indicator of the Social Capital of an enterprise and how an enterprise gains a favorable advantage through the direct or indirect use of social connections. For our concern, we are looking at digital social connections (facebook, myspace,  linkedin, blogs etc)

As enterprises move towards the social computing era and embrace social networks and Web 2.0 components (blogs, wikis, discussion forums, twitter etc), need for managing your relationship with the social computing environment is becoming a necessity.

As Enterprises start embracing the Social Networks (myspace, linkedin, facebook or orkut) and integrate with thier corporate web sites, enterprises start need to be aware of the following -

Google has announced that they will be launching Chrome OS soon. All the analysts are predicting how it will be competing against Microsoft's Windows or how it will power the netbooks with more features available off the web + secure environment.

What I am visualising is the advent of Chrome OS with a small kernel and Chrome browser kind of inteface.

Singleton is a design pattern means allowing only one instance of the class. (Check out more on singleton’s here) Singleton works well till the point you have single JVM.

In a multiple JVM’s environment, each of them will have their own copy of the singleton object which can lead to multiple issues specially in a clustered environment where the access to the resource needs to be restricted and synchronized.

To achieve clustering across JVM’s, one can use multiple techniques (JMS, DB, Custom API, 3rd party tools), but each of them have an impact on the business logic. If the requirement for JVM clustering is realized later in the game, then change to business logic will be huge.

• In case, the requirement to have singleton across JVM’s is realized later, then tools like Terracotta, Oracle Coherence are good options. These work on the concept of providing an in memory replication of objects across JVMs in effect providing you singleton view  or making use of any of the cluster-aware cache provider’s like SwarmCache or JBoss TreeCache should work as cache entries are singletons and clustering is built in.
• Also, there is product called JGroups – which uses multi-cast comm. (TCP/UDP).  It allows to form a group and Applications (JVM’s) can participate and JGroups will send messages to everyone in the group so that they can be in sync. 
• Application server’s also provide some level of custom API’s to circumvent this problem. 

1. JBoss has HASingleton Service ( based on MBeans) which is meant to solve this problem. Check here and here
2. Weblogic has the concept of Singleton Service – where only instance runs within the cluster and all clients will look up to the same instance. 
3. WebSphere supports the concept of singleton across cluster in the WebSphere XD version of the application server as the partition facility – ObjectGrid 

How many times I have heard the statement “Code should be maintainable” from clients, Delivery Managers or Project Managers. I always ask them, how you quantify maintainability. I am listing down some of the factors that can be measured and help make the code maintainability quantifiable.

• Code Coverage– describes the degree to which the source code of a program has been (unit) tested. Popular tools to measure code coverage in Java is Emma. You can find more code coverage tools list here
• Code Complexity - is another measure that can used to identify and quantify the code maintainability. Factors like Cylometric Complexity, Halstead's software science metrics and Lines of code metrics are some of the factors. Infact, there is a measure called Maintainability Index that can be calculated by using the factors. Check out tools to calculate MI here.
• Documentation - is another factor that be used to quantify code maintainability. Documentation can be your normal JavaDocs or the inline source code commenting. You have tools like JavaNCSS where NCSS ( non commeting source statements) that can be applied at class, method/function level
• Code Format - is not really a measure but is an indicator of the readibility of the code.Tools like CheckStyle can help identify the code formatiing problems (from non standard class headers, identation issues to spacing, variable naming convention and so on). Any issues reported by the tool can be used an indicator of the maintainability of the code

IBM has introduced the full set of features from Lotus Connections (or inhouse IBM Blue) to the Developerwork member community. Now, one can enjoy the full social networking features available as part of Lotus Connections (Profiles, Dogear, Communities, Blogs, Activities) as part of the DW membership.

Check out a sample My developerWorks : Home Page

Check out my profile page here and do tag my profile.

I have moved from blogger to my own domain - http://www.techspot.co.in. Nothing else has changed !

IBM as part of the User Experience Optimization Initiative has releases a white paper on integration of WebSphere Portal with Omniture. The integration helps in tracking the following

• What pages/site sections receive the most views?
  
• What is the affinity between pages and site sections?
  
• What content drives further site consumption?
  
• What pages/site sections exits and entries?

The Omniture integration makes use of Dojo libraries that are now bundled with WebSphere Portal V6.1

For WebSphere Portal integration with Omniture SiteCatalyst, you will need 2 files - s_code.js and specific dojo analytics class created specifically for the WebSphere Portal integration. The javascript file is placed in the theme folder and referenced within the default.jsp.

In addition, if the requirement is to track the specific events (or special events that need to triggered on user button clicks), one can edit the _eventRegistry variable found in analytics.js to add listeners to specific buttons and links on the site.

The integration is support only from WebSphere Portal V6.1 onwards.

Check out the white paper here .

In WebSphere Portal V6.1, there is a new feature available called the Web 2.0 theme. 

Till the previous version, all portlets on a page use to get the rendering aggregation done at the server side-also called Server Side Aggregation(SSA). 

The Web 2.0 theme allows what is called the Client Side Aggregation(CSA). The CSA is made possible by using AJAX. The CSA provides definite advantages over the SSA

• Faster rendering and Performance - since the theme is already pre-compiled and ready, the theme gets rendered to the browser immediately. From the browser, requests are invoked for individual portlets. Overall, gives the impression of faster loading of page to the user. Also, as the aggregation does not happen at server side, load on server also comes down
• Better user experience - since with CSA, the entire page need not be refreshed, only the relevant portlet area is refreshed where user activity takes place. Giving a better user experience

For the enterprises that are planning to migrate to WebSphere Portal V6.1 and want to make use of the Web 2.0 theme, the existing portlets that can used with the theme and they can also make use of AJAX calls for getting their portlet area's refreshed instead of the whole page.

For the user's who may not have JavaScript enables, the theme defaults to Server Side aggregation for such scenario's.

To take advantage of all the functionality provided by CSA, IBM has introduced Client Side Portlet Programming model. The Programming model provides additional tags to handle portlet preferences change, changing portlet window state, changing portlet mode on the client side itself.

When developing Portal applications, there is a tendency to pit too many functionalities on the portal node. There are easier ways by which we can reduce the load on the portal node by following some of these guidelines

• Avoid running multiple services on the same portal node. e.g. Set up Search on a different node or serve images from a dedicated server ( can be edge server)
• Avoid synchronising session data across portal nodes. Sync user session data using Dynacache is huge overhead on the CPU and is very memory intensive. Try using sticky sessions or avoid storing huge data on session
•  When using WCM off the portal node, see if you can make use of remote rendering instead of local rendering
• Try bundling logical portlets in the same EAR. Helps to avoid multiple classloaders
• Uninstall the portlets that are not being used in the environment. If you are managing the environment using XMLAccess, one can even uninstall the adminstrative portlets
• Cache as much as possible - whether it is WCM content or portal assets
• If you deploying newer portal applications - see if you can make use of Client Side Aggregation (CSA) 

Check my earlier post Why Portals might be crawling??

I have recently been part of one big bang project. The project needed to go live in 12 months with everything to be put in place. When I say everything, it means the RFP needs to be floated, the vendor needs to be selected, the architecture needs to be defined, infrastracture needs to be identified, ordered and set up. Even the Data center's needed to be defined. The application itself was very large. One could imagine every problem and it would had manifested in the program one way or the other. 

The business mandated the tight timelines and schedules in order to beat the hell out of competition. The organisation IT dept did advised them not to go for a big bang.

In the end, the deadlines were missed and with the economic recession storm coming down, the business priorities changed, the program was put on a back burner with reduced funding and newer timelines.

My Qs, whom did the program helped ? Neither the organisation nor the your customers. Yes, the vendors did made the money selling hardware boxes and software licenses and services.

So, why do we keep going after these treasure hunts again and again.

So, why do not we identify the business priorities, look at the competitive and technology landscape and start working on creating value at a smaller pace but something more sustainable.

Most times, business has vague ideas about what they want, but IT can become an hinderance in the implementation (need for budgets, lack of people, need to buy newer stuff, build vs buy etc)

, so what it requires is IT dept to be more tuned to the needs to the business, be aware of the happenings in the market, keep an eye on which ways the applications are going and be ready to serve the needs.

Today, the people in IT dept are the biggest impedients to the adoption of new technologies. They have hundered excuses - need for budgets, lack of people, need to buy newer stuff, trainings and what not. 

Why do they not go that extra mile to constantly upgrade their skills, learn new technologies and be ready for next business requirement. 

Why do they not work like IT consultants who constantly keep upgrading themselves to stay above the curve and not make themselves redundent. 

Because if you do not, you will see your work getting outsourced to people who are far more hungry to prove themselves. Maybe they do not have 20 years of experience behind them,  but with every passing year, the number of experienced people are going up and soon you will not be able to hold on.

When one looks at the Architecture Frameworks available all over, one can easily identify a common pattern. All the common architecture frameworks revolve around the same 4-5 different views. 

So as long as one is taking into account the below mentioned views, the architecture can be fitted into any of the known architecture methdology.

Do you think, there is anything other view, an architect needs to be worried about. Do write in !

Web2.0 Ajax and REST in WebSphere Portal

View more presentations from Munish Gupta.

Check out javadoc's for the WebSphere Portal V6.0 and V6.1 programming interface specifications here.

Most of the Web Applications, where there is a feature for file upload, there is a requirement for doing virus scan of the uploaded file. When I checked around, there were not too many options in the FOSS area. 

Antivirus scanners are available in the FOSS area but integration with the web application is still a nightmare. Check the list of antivirus software here.

I did some googling around and eventually came up with the following integration options 

1. Command Line Option - Use a command line virus scanner,use the Runtime class to execute a scan on a file and based on the output of the program, display to the user whether or not the file is clean.  Sligthly tricky and not a very clean program but still does the job. 
2. Using OPSWAT Meta Scan - OPSWAT MetaScan is an advanced engine for launching scan requests and obtaining scan results from many antivirus applications, as well as for analyzing file types. Metascan enables IT to develop multi-scanning solutions leveraging built-in antivirus engines from AVG, CA, ESET, Norman, VirusBuster EDK, MicroWorld, and ClamWin. Additionally, Metascan includes an easy, scalable interface to integrate with almost any antivirus engine on the market. Clean and elegant solution, but need to write a check for the same
3. Using ModSecurity - ModSecurity is a Web Application Firewall that can be configured with the Web Server. ModSecurity has a feature to for inspecting files on upload which can be combined with an antivirus scanner software. Again not something out of the box. You end up adding another product for integration
4. Using ICAP Servers  - ICAP is a protocol designed to off-load specific Internet-based content to dedicated servers, thereby freeing up resources and standardizing the way in which features are implemented. Check out ICAP-Server for an Open Source implementation of ICAP server. One can run GreaseSpoon on top of an ICAP server. Greasespoon can intercept the HTTP traffic and allow the application to perform the virus scan. Again a convuluted way of achieving the same. SafeSquid is a another product in the market 
   

Hope it helps others. Also, if someone has found another product/integration technique, please do let me know.

Amazon has released the AWS Toolkit for Eclipse, which is a plug-in for the Eclipse Java IDE that makes it easier for developers to develop, deploy, and debug Java applications using Amazon Web Services. 

Just goes on to show the commitment on Amazon's part to provide tools to help develop application on EC2.

Read more here.

There are series of new articles available that detail how to use JiBX for converting Java to XML and vice versa. Very good for anyone who is starting to work with JiBX.

Check them out here.

Part 1

Part 2

Since, it is possible for a single WebSphere Portal HTTP request to use multiple database connections concurrently, the ratio of the data source connection pool size to the WebContainer Thread Pool needs to be tuned.

For the WebSphere Portal databases, the recommended ratio is 2:1.

For Workplace Web Content Management (JCR) database, the recommended ratio is 2.5:1.

For example, if the WebContainer thread pool maximum is 50, then each WebSphere Portal data source connection pool maximum should be 100 and the JCR data source connection pool maximum should be 125.

Verify that your database server is able to handle all incoming client connections from all WebSphere Portal cluster member(s).

Check out the Tech Note here.

Enterprises initiate large multi-year, multi-million programs that meant to overhaul the existing applications, provide more flexibility to business managers, and provide more features to the end customers and so on. As the statistics speak, 2 out of 3 IT programs are a failure.

Now, the question is who pays for the failures of these programs?

It is the end customer. People like you and me.

The enterprises will load these failure costs as part of the operational costs and in the end, all these get charged as the cost of the service provided by the enterprise.

Click on the image to see an enlarged view

IBM has released Amazon Machine Instances (AMI's) for the WebSpher Portal V6.1 and Lotus Web Content Management. There is an AMI also available for WebSphere sMash V1.1. 

The AMI's allow the developers to develope portals using the Amazon Services. Idea is to utilise the existing investment that an organisation might have already made on applications that already hosted on the Amazon Web Services (AWS). 

Some of the area's where the Portal makes sense is

* developing self contained web sites

* aggregation on EC2 ( Amazon Elastic Compute Cloud)

* aggregation of robust  web services

Check out more details here.

Consider using SOAP over HTTP for:

• Externally facing web services (e.g. B2B scenarios) 
• For simple point-to-point and stateless services 
• Where you need a thin client with no message oriented middleware installations 

Consider using SOAP over JMS for: 

• when you require high-volume distributed messaging 
• Asynchronous messaging 
• Where a transaction boundary is needed in the middleware 
• Where the message consumers are slower than the producers 
• Guaranteed deliver and/or only once delivery of messages 
• Publish/subscribe mode
• Distributed peer systems that might at times be disconnected 

IBM is working on an initiative to provide a framework that dynamically integrates and synchronizes people, processes and assets associated with software development projects. Jazz is an initiative to transform software delivery making it more collaborative, productive and transparent.

• Jazz separates the implementation of tools from the definition of and access to the data. 
  
• Jazz federates data across independent databases using Internet protocols. 
  
• Jazz can access and integrate data where it resides - Jazz does not need to import and export data between tools or repositories
  
• Jazz assumes an open, flexible, distributed data model. 
  
• Jazz allows tools to be implemented in any Internet-aware programming language or platform. 
  
• Jazz supports multiple client technologies. 
  

The Jazz Integration Architecture (JIA) is based on the industry initiatice called Open Services for Lifecycle Collaboration (OSLC). At the center of the Jazz Integration Architecture is the Jazz Team Server (JTS). The JTS implements the Jazz Foundation Services described by the JIA, which enable groups of tools to work together. These services include user and project administration, security, collaboration, query, and other generic cross-tool capabilities. When installed, tools are associated and work in conjunction with a particular JTS.

Jazz is being developed as an open intiative. More details on how to download and start using the jazz platform can be found here.

Wipro Web2Works is a Mashup framework that enables rapid synthesis and rollout of new business services. It is based on the gadget/widget philosophy of empowering communities to create, publish and mix content, using a standard Web browser. It enables semi-technical business analysts to rollout new lines of services and customized interfaces by simple drag-and-drop. It enables end users to personalize their Web Interfaces per individual preference. 

Festa is a reference implementation of Web2Works (W2W). 

Check it the framework and the reference implementation out here.

Download a free trial version of WebSphere Portlet Factory V6.1.2 and experience firsthand how Portlet Factory's ease of use and advanced development features supercharge and dramatically streamline the entire portlet, widget, Web and rich client application development process.

List of the most Recently Added Content to the Websphere Portal support site.


Administrative pages are missing in virtual portals
The administrative pages that should contain the administrative portlets do not exist in a virtual portal created in an admnistration installation (as opposed to a full installation)

After clustering Portal, all ConfigEngine scripts end with a "NullPointerException"
After clustering IBM® WebSphere® Portal v6.1, all ConfigEngine scripts end with a "NullPointerException" regardless of whether or not you receive a BUILD SUCCESSFUL.

Single Sign-On fails with SiteMinder due to incorrect agent group settings
After configuring eTrust SiteMinder for WebSphere Portal, you still get prompted for the portal server login after authenticating via the SiteMinder login feature.

Incorrect providerURL causes ConfigEngine wp-add-property to fail in cluster
When adding an LDAP attribute by running "ConfigEngine.bat wp-add-property", the task fails with the following errors: [wplc-add-property] [12/20/08 08:32:43.951 EST] Looking up EJB: corbaloc:iiop:localhost:10031... [wplc-add-property] [12/20/08 08:32:44.105 EST] security.LoadSCI [wplc-add-property] [12/20/08 08:32:44.116 EST] security.GettingConfig [wplc-add-property] [12/20/08 08:32:44.128 EST] security.AuthTarget [wplc-add-property] [12/20/08 08:32:44.185 EST] security.ClientCSI
Installation and Configuration Troubleshooting
This document describes solutions to some problems encountered in the areas Installation, Configuration, and Migration.

XmlAccess export fails with EJPSG0015E due to portlet instance in customization domain
When performing a full XmlAccess export using export.xml, you receive one of the the following error messages: com.ibm.wps.command.xml.XmlCommandException: EJPXA0033E: An error occurred while retrieving the configuration of the resou

Member Manager cumulative fix for WebSphere Portal version 6.0
Includes all previous WebSphere® Member Manager fixes for IBM® WebSphere Portal version 6.0 released interim fixes plus new fixes. This document references the current version.

How to avoid removing all mappings from a role via Xmlaccess
How can I remove a user or group mapping from a role on a resource via Xmlaccess without removing ALL mappings for the role? For example, I wish to remove "mygroup2" from the User role on the myPortlet portlet but retain "mygroup1". Currently my Xmlaccess export of this portlet shows:

Expected behavior when one repository is down in a federated environment
When configured for multiple (federated) repositories in a WebSphere Portal environment, should a user in repository A be able to log into the portal server if repository B is down?


CWWIM5015E error when configuring security
When running the ConfigEngine task "wp-modify-ldap-security" to set up a stand-alone LDAP user registry for WebSphere Portal, you receive the following error: Create LDAP repository - result: [com.ibm.websphere.wim.exception.WIMConfigurationException: CWWIM5015E Login properties are not valid: [uid] . . . . [wplc-create-federated-ldap-repository] Create LDAP repository failed.[]

Implementing the timeout.resume.session parameter in WebSphere Portal
When should the timeout.resume.session parameter be implemented in IBM® WebSphere® Portal, and what is the expected behavior when setting this value to "true"?

MustGather: Session Management for WebSphere Portal 6.0
This technote covers collecting data for Session Management with IBM® WebSphere® Portal 6.0. Gathering this MustGather information before calling IBM Support will help to understand the problem and save time analyzing the data.

Configuration task disable-security fails with "x'@EncryptedPortalAdminPassword@'" is not valid. SQLSTATE=42604
The IBM® WebSphere® Portal WPSConfig task 'disable-security' fails with SQL0105N The string constant beginning with "x'@EncryptedPortalAdminPassword@'" is not valid. SQLSTATE=42604 Full Error message in the SystemOut.log: [sqlproc] Error during statement execution! [sqlproc] COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver][DB2/NT] SQL0105N The string constant beginning with "x'@EncryptedPortalAdminPassword@'" is not valid. SQLSTATE=42604 [sqlproc] at COM.ibm.db2.jdbc.app.SQLExceptionGenerator.thro

Database DB2 transfer (. /WPSconfig.sh create-local-database-db2) fails
All of the prerequisite checks in the IBM® WebSphere® Portal create-local-database-db2 script pass successfully but when it gets to the actual creation of the database in DB2® it fails with the following: Could not load driver java.sql.SQLException: java.lang.UnsatisfiedLinkError: Can't find library db2jdbc (libdb2jdbc.so or .so) in sub.boot.library.path or java.library.path sub.boot.library.path=/usr/IBM/wp_6/WebSphere/AppServer/Java/jre/bin java.library.path=/usr/IBM/wp_6/WebSphere/AppServer/Java/jre/bi

DB2 Rights for the Portal Runtime Database User
Many users install IBM® WebSphere® Portal version 6.0 using an administrative user with DBADM rights, but prefer to configure a runtime database user with a subset of the administrative privileges. Since there are several databases used by WebSphere Portal v6, what are the runtime user requirements for each database?

Portal installation fails when hostname contains underscore "_"
During the installation of IBM® WebSphere® Portal, you receive a failure notice, stating "ADMG0005E: The attribute name is not valid."

MustGather: Read first for IBM WebSphere Portal
MustGather documents aid in problem determination and save time resolving Problem Management Records (PMRs).

I recently saw this list of delivery guidelines which any project team should adhere to. Not adhering to these will only lead to burning yourself and potentially projects

• Always identify you risks (technical are most important which you can take care of)
  
• Always start early coding (say after 2 weeks of project start. Sometimes it takes months for your team to configure project in a specific IDE and server)
  
• Risks are not only yours. Let your customer and other stack holders know about them. Risks include availability of resources and let responsible people know about it early.
  
• No process can help non performers or unskilled resources to write quality code. Competency development is the responsibility if others and running project can not training people from scratch so evaluate your resources properly and let people know that your are not going to do miracle with unskilled resources.
  
• Re estimate and don’t try to manage the things if estimate says more effort is required. No process or manager can make it possible to achieve 100 man days task in 80 man days. Let people know more effort is required and schedule will certainly be impacted if you don’t have extra resources.
  
• Be practical on colander. If an assignment requires 1 calendar year it can not be completed in 3 moths by just adding 3 times more resources.
  
• Without having requirements written and testing done (unit and system testing) no process can help you passing acceptance testing.
  
• Always include end users (business users) in requirements discussions and get their signoff also. No one except end users can give you and idea about their real requirements.
  
• Don’t expect end users to give you specifications. Give them early demo of wire frame, prototype or early code so that gaps can be filled before death.
  
• Delivery is the responsibility of delivery team not quality department so don’t expect much from processes. Quality and processes are support functions. Know your job better, plan it and do it. If your customer is happy everyone will be happy.