Clustering the LCM and VMS in SAP BusinessObjects 4.0 SP4

With the release of SAP BusinessObjects Enterprise 4.0 (BOE 4) and subsequently with the release of Service Pack 4 (Including Feature Pack 3) administrators will find that both the Life Cycle Manager (Promotion Management or LCM) and the Version Management tools are included with the installation. In XI 3.1, when these tools were first released, a separate installation was required and it was considered an add-on to the platform. With BOE 4.0 they are included with the main server installation media. As of BOE 4.0 SP4 (with FP3) these tools are directly integrate into the Central Management Console (CMC). For single server (Full installation) deployments there is very little you need to know about the interworking of these tools. However, if you plan to deploy BOE 4.0 SP4 as a distributed or clustered environment there are a few tricks to successfully deploying LCM in a clustered environment. This document will serve as a means to explain the interworking of the LCM within a cluster and the steps to deploy LCM as an active / passive service.

Before we discuss the steps of procedures for clustering, let’s break the LCM down into its basic components.

The LCM Web Applications:

There are two main tools that encompass the LCM and VMS tools that are managed at the Java Web Application Server (JWAS) level. In SP4 you can access them via the CMC under “Promotion Management” and “Version Management”. There are also a few settings for these applications found within the “Applications” area of the CMC. The two main settings are for the SVN services (found under Version Management) and the overrides database (found under Promotion Management). Because these tools are a component of the CMC and run under the JWAS (Tomcat in most cases) they can be deployed on one or more JWAS (Tomcat for example) servers within a cluster. SAP does not provide a solution for clustering Web Servers out of the box, but you will like find that your network team already possesses a tool to help you cluster Web Servers. Typically organizations will utilize an IP Load Balancer (with sticks sessions) to cluster web servers but there are other options as well.

Subversion:

“Subversion” is a software versioning and revision control system distributed under an open source license. Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly-compatible successor to the widely used Concurrent Versions System (CVS).“ Reference . The LCM and VMS applications within BOE 4.0 will utilize this service to maintain snapshots or rollback instances of BOE content. Subversion is installed automatically with the main BOE 4.0 installation media as a 3^rd party component. It will run on any node within your cluster and typically listens on port 3690. The service itself manages a file based repository. The default repository name is LCM_repository and its data will be stored on the local file system where Subversion is installed. When deploying a clustered environment it is best to only deploy subversion on one node. This will make it easier to migrate the repository when we start to cluster the services. The main executable for the SVN service is located in the subversion\bin directory found under the BOE 4.0 Windows folder “Program Files (x86)\ SAP BusinessObjects\”  or UNIX folder “/sap_bobj/enterprise_xi40/”. The LCM_repository folder will contain a file based database that tracks the version, location and other information about files that are being managed by the LCM and VMS tools.

The Check_out Directory:

The check_out directory is another file based repository or database that maintains an actual copy of each BOE 4.0 object that is either promoted or versioned using the LCM or VMS tools. If you browse the directory you will find subfolders named after each objects CUID. Within each of these subfolders you will find a small file based db and a series of .bair files that represent each object or object promotion. When establishing a cluster this directory must be moved to a central NAS share utilizing either SMB or NFS. Later we will configure BOE to look for these files under a shared UNC path or shared
NFS mount.

The LCM Overrides Directory:

The LCM Overrides directory is new in BOE 4.0. It is another apache derby file based database that is utilized to track the translation of connection, QaaWS and Crystal Reports connections from one system to another system. Again, when we cluster the LCM / VMS services we have to also migrate this database to a central shared file system (NFS or SMB).

The LCM Adaptive Processing Services:

The LCM Adaptive Processing Services consists of three components:

1. Lifecycle Management ClearCase Service
2. Lifecycle Management Service
3. Visual Difference Service

In a properly configured environment you will undergo an exercise to split the default AdaptiveProcessingServer into multiple dedicated Java services. When doing so it is important to create a dedicated APS_LCM service that contains the above mentioned LCM sub-services. This will be important for LCM clustering as we will need to stop and start these services as a group on each node. The Lifecycle Management Service is very important to the management of the LCM / VMS processes. It is responsible for communicating directly to SVN repository, the Check_out and Overrides repositories.

Clustering the LCM (Overview):

Clustering the LCM is not a straight forward process so pay close attention to the steps and requirements. In short, because the LCM relies on file based databases or repositories there are restrictions to having multiple active services managing these files. File locking issues can be a real problem when two services are actively accessing the same file. As a result we have to run the LCM clustering in a true active / passive (manual passive) fashion. This means that both the subversion service and the LCM APS services can be running on one and only one node at any given time. It also means that they must be running together on the same node at any given time. The other main requirement is that we move the three main file based db repositories to a central shared file system. On Windows this can be an SMB UNC path or on Linux this can be a shared NFS mount.

Clustering the LCM (Steps):

Step 1: With the VMS and LCM working on Node 1, go tothe CMC, Version Management tool and add 1 or more reports to version management. This will help with subsequent testing as objects will be added to the repository.

Step 2: Stop both the LCM_APS (Adaptive Processing LCM Services) and the Subversion service on all nodes.

Step 3: Insure that the subversion services were installed on all nodes where it can potentially run. If the services were not installed you can add it again by re-running the BOE install media. For linux environments you can create two svnstartup.sh and svnshutdown.sh scripts to make the process of stop/starting easier (See SAP NOTE: 1755733 for Example). On Windows you will find the Subversion services listed under the standard Windows Services MMC. On Windows you can also create a .bat file to start / stop the services if you so desire.

Step 4: Copy the “Check_out”, “Overrides” and “LCM_repository” directories to a central shared file system. I recommend removing / deleting the original directories once you have everything working. I placed all the directories in a single path. For example:

Windows: “\\servername\sharename\lcm”

Linux: “\apps\bobj40\shared\lcm\”

Step 5: Update the SVN root path and then start the Subversion service on Node 1. Using information found on the http://subversion.tigris.org/ site, insure that the SVN services recognize that the LCM_repository now resides on the shared file path. In my environments, I simply update the SVN startup to include the shared file system as the new SVN root. If the repository directory was copied correctly and the SVN services are started with the correct root path, the LCM_repository should be recognized automatically.

Unix: “./svnserve –root=\apps\bobj40\shared\lcm\ –port “3690”

Note: Do not include the LCM_Repository folder in the root path. On UNIX the patch and repository name are different then on Windows.

Windows:

1. Using Regedit (with Caution) Go to [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\services\SVNSubversion]
2. On the ImagePath String REG_EXPAND_SZ, update the path following the –r command.
3. Change -r “<Install Path>\SAP BusinessObjects Enterprise XI 4.0\LCM_repository” to -r \\server\Share\lcm\LCM_repository . Note: Include the LCM_repository directory in the above path. On Windows the SVN repository is in a sub folder named svn_repository.
4. Go To the CMC, Applications, Version Management, VMS Settings and make sure that the repository name field is set to  svn_repository

Step 6: On the remaining nodes (that will also potentially run the SVN services) update the subversion startup scripts or registry to make sure that they also will use the new SVN root on the shared drive when started. Stop and start the SVN services on each node (one at a time) and verify that the services are working and pointed to the correct SVN root directory. When the following command is executed on each node, you should see a list of folders matching the CUID of the objects that were checked in during step 1. You should be able run the following command on each server to verify.

Command:

• (UNIX):    svn list svn://localhost/LCM_repository
• (Windows) svn list svn://localhost/svn_repository

Note: The SVN can be found under the SAP BusinessObjects Installation directory. The paths below might be different in your environment:

Windows: <Install Directory>\SAP BusinessObjects Enterprise XI 4.0\subversion

Linux: <Install Directory>/sap_bobj/enterprise_xi40/subversion/bin

Step 7: Update the version management settings by going to the CMC, Application, Version Management and Version Management Setting Page. Set the options as follows:

1. Server Name:    localhost <-default (must be localhost for this solution to work)
2. Server Port:        3690 <-default
3. User Name:        lcm <-default
4. User Password: <do not change>
5. Install Path: <Path to the “subversion\bin” directory> <-default
6. Repository Name:           LCM_repository  <-default Unix   svn_repository <- default Windows
7. Workspace Directory: <change to the new shared directory path of the “Check_out” directory.
8. Protocol: SVN

Note: localhost is required. When objects are added the the check_out directory, the repository records the host name of the subversion service’s host. If these settings list an individual host, the fail over will fail because a different host name will be recognized.

Step 8: Update the “Overrides Directory Path on the APS_LCM services. Go to CMC, Applications, Promotion Management, LCM Override Settings. For each APS_LCM services configure the path for the overrides database to the shared directory location of the “overrides” folder.

Step 9: On one and only one node, start the Subversion and APS_LCM services. Insure that these services are stopped and disabled on all remaining nodes.

Testing the LCM failover:

At this stage you can either test the LCM failover or conduct an actual failover by stopping both the Subversion and LCM_APPS services on NODE1 and subsequently starting both the Subversion (first) and LCM_APS (Second) services on NODE2. The key is making sure that both of the services are running on the same NODE and the same time. You also need to make sure that these services are disabled and stopped on all remaining nodes.

Hopefully this will give you more insight into the process of clustering the LCM services and databases. Each environment will be different so it is better to first understand the processes and dependencies and then tailor this configuration to your environment.