Error while creating new jobs in DataStage

If you face the error error similar to below can occur when trying to save a newly created job:

Error On CREATE.FILE command
Creating file "RT_CONFIG4817 as type 30, mkdbfile: connot create file
RT_CONFIG4817. Unable to create operating system file "RT_CONFIG4817"

Get list of activated tools in Datastage Installation

If we want to check what features have been installed with datastage, we can check it by using LicensingServiceAdmin  tool. This will list down all the tool packs or features including Qualitystage as it comes as a part of datastage.


Tool Location :  {IS_INSTALL_DIR}/ASBServer/bin

Maximum number of DataStage Jobs in a project

Filesystems can have a hard limit on number of subdirectories that can be contained in a directory, for example 32766 (32K -2) for Linux

For every job, DataStage will create hash files as subdirectories in the project directory. Here is a list of the subdirectories created for every job. In this list "nn" represents the number (or id) of the job:

DS_TEMPnn

RT_CONFIGnn

RT_STATUSnn

RT_LOGnn

RT_BPnn

RT_BPnn.O

RT_SCnn (only for parallel jobs)

Unlocking XMETA locks from jobs

How can I unlock an XMETA locked job?

The job cannot be exported and when attempting to open the job, the following error appears:

Unable to open the Job
The Job 'TestJob' is locked by user 'dsadm'
(SessionID '8F1B0326-4F7E-45A8-B8EC-EAA8C4780528')

Do you want to continue and open a read-only view of the Job?

This issue arises when a lock still remains in XMETA for the corresponding job. Information Server will lock the job to ensure that there is no indeterminancy regarding the editing and saving of a job.
The most common causes of this is when a user has closed a client window unexpectedly (e.g. Force Terminate).

DataStage Installation Checklist

• Review the system requirements.
• learn about tiers and components.
• Design the topology of your installation.
• Determine whether to pre-install your database system.
• Determine whether to pre-install IBM WebSphere Application Server or pre-configure an existing installation of WebSphere Application Server.

Error - mongod shell doesn't start, /data/db doesn't exsist

Error - mongod shell doesn't start, /data/db doesn't exsist


If you face this error while starting MongoDB (mongod), most probably there are 2 reasons for this -

a.  /data/db is not exist at all
b. mongod process don't have access on /data/db

Solution :-

The default db path for mongodb is /data/db/.
So when you run:

Encrypt Password for DataStage Command Line

As datastage user,  we need lots of scripts help our daily  jobs,  most of these scripts need authenticated by datastage engine,  that means we need to pass our Datastage user password in script or command line. 
                     But there is a problem if we are using "Password" here, It can be view by checking the content of script or command history. To secure password, there is a way provided by datastage and that is ENCRYPT the password before using. Let's see how we can do that ---

Windows Command Prompt setup for DataStage Command Line

As we know that for using Datastage Command Line in Linux system, we need to setup linux shell session likewise we need to do same for Windows if server is installed on Windows server.

Follow below steps to setup your Datastage command Line ---

DataStage Error - Cannot authenticate user

When you attempt to start one of the InfoSphere DataStage and QualityStage Administrator clients, the following message is displayed:

Failed to authenticate
the current user against the selected Domain:  Invalid user name (username)
or password.

 

There are several possible causes of this problem.

• The user name is invalid.
• The password is invalid or has expired.
• The user has no suite user role.
• Credential mapping is required, but has not been defined for this user.
• The user has no DataStage role or has the incorrect DataStage role.

Linux Shell session setup for DataStage Command Line

To setup Linux Shell for Datastage command line is very important if you are a DataStage Developer and want to work hassle free on DataStage Command Line.

Follow below steps to setup your profile ---

a) First, we will create .dshome file which is containing the path of DS Engine. If you know what it is, good, create a file in root dir with DS Engine path
otherwise execute below scripts to do it by itself

Oracle table is Partitioned or Not ?

For getting whether Oracle table is Partitioned or not, we can use below query on ALL_TABLES -

WAS server location for IBM Information Server

How to find Correct WAS server using by your Information Server ?

IBM Information Server uses a Websphere Application Server. If you have multiple WAS servers installed, to find the one that IIS is using you can look in:


C:\IBM\InformationServer\ASBServer\bin\setupEnv.bat
or
/opt/IBM/InformationServer/ASBServer/bin/setupEnv.sh

Navigating the many paths of metadata for DataStage 8

Source : Navigating the many paths of metadata for DataStage 8

Looking at the methods for importing metadata table definitions into DataStage 8 ETL jobs.
All of the metadata import methods of DataStage 7 are in DataStage 8 and all execute in the same way.  Developers familiar with previous versions will be right at home!  What is tricky to come to terms with are all the new ways to get metadata into DataStage 8 and the Metadata Server.  The Metadata Server can provide reporting on metadata from products outside of DataStage such as BI tools so in some cases you might be importing the metadata for reporting and not for ETL. 
The list that follows covers just the techniques for importing metadata to be used by DataStage jobs.

APT_CONFIG_FILE : Configuration File

APT_CONFIG_FILE is the file using which DataStage determines the configuration file (one can have many configuration files for a project) to be used. In fact, this is what is generally used in production. However, if this environment variable is not defined then how DataStage determines which file to use?

1)If the APT_CONFIG_FILE environment variable is not defined then DataStage look for default configuration file (config.apt) in following path:

1)Current working directory.

2)INSTALL_DIR/etc, where INSTALL_DIR ($APT_ORCHHOME) is the top level directory of DataStage installation.

What are the different options a logical node can have in the configuration file?

Some Oracle SQL should known by Developer - 1

========================================================================

Some more design tips for DataStage Job Development

• Common information like home directory, system date, username, password should be initialized in a global variable and then variable should be referred everywhere.

• Stage Variables allow you to hold data from a previous record when the next record, allowing you to compare between previous and current records. Stage variables also allow you return multiple errors for a record of information. By being able to evaluate all data in a record and not just error on the first exception that is found, the cleanup of data is more efficient and requires less iteration.

DataStage error upon login: DSR.ADMIN: Failed to add user to UV_USERS file

When user first logs in, they get following error message:
DSR.ADMIN: Failed to add user to UV_USERS file.
ACCESS DENIED: INTERNAL SQL ERROR FAILED TO REGISTER THIS USER FOR SQL.

Get list of locked item on DataStage Server with Script

This script can be run with or without any argument and provide the list of Job, item locked in DataStage environment by any user. This script will help release the locks if user doesn't locked the item.

Script Name : DSgetJobLocks.sh
Usage : ./DSgetJobLocks.sh [one or more argument]

DOS Batch Script to Export DataStage Jobs Automatically from a Project

Copy the following script into a text file and rename into ".bat" file.

 This batch script is used to Export all jobs from a Projects
 This script must be run from a DataStage client machine and the parameters below should be given

 Host is server name
 User is DataStage username
 Password is DataStage password
 Imp Location is the directory where the datastage dsx's are stored

Check whether DataStage Job is Multi-Instance or not with Sctipt

The following script can be used to check if the given datastage job is multi instance or not.

Arguments to the Scripts:

Arg1:Datastage Project Name
Arg2:JobName