Error while creating new jobs in DataStage

If you face the error error similar to below can occur when trying to save a newly created job:

Error On CREATE.FILE command
Creating file "RT_CONFIG4817 as type 30, mkdbfile: connot create file
RT_CONFIG4817. Unable to create operating system file "RT_CONFIG4817"

Maximum number of DataStage Jobs in a project

Filesystems can have a hard limit on number of subdirectories that can be contained in a directory, for example 32766 (32K -2) for Linux

For every job, DataStage will create hash files as subdirectories in the project directory. Here is a list of the subdirectories created for every job. In this list "nn" represents the number (or id) of the job:

DS_TEMPnn

RT_CONFIGnn

RT_STATUSnn

RT_LOGnn

RT_BPnn

RT_BPnn.O

RT_SCnn (only for parallel jobs)

Script to UnLock DataStage Jobs

This script will help you to unlock the all jobs, if you are a DS admin, or your datastage jobs , if a DS developer, without doing human error while executing command on Datastage server UV shell or Datastage Administrator Client.

Script is quite simple, Execute it , it will ask for Project Name and Job Name and Done if you have appropriate access on Datastage server.

May be need a modification as per your Datastage platform or if you need some fancy stuff to add on.  Happy Scripting !!

Get Datastage Job Lock List by Script

** This script will fetch all locks currently on Datastage Server irrespective of Projects.
** We can modify this script to correct the data display format as per our need
** Can modify sort command as it changes its behavior in different Linux flavour

DataStage Error - Client logins fail with Error: 39125

DataStage client logins fail with error: 39125: the directory you are connecting to either is not a uv account or does not exist.

This error usually indicates that one of the 6 files required to make a UniVerse account is missing from the directory of the DataStage project which has the problem or that the user does not have permission to access these files. Those 6 files/directories are:

DataStage error upon login: DSR.ADMIN: Failed to add user to UV_USERS file

When user first logs in, they get following error message:
DSR.ADMIN: Failed to add user to UV_USERS file.
ACCESS DENIED: INTERNAL SQL ERROR FAILED TO REGISTER THIS USER FOR SQL.

Get list of locked item on DataStage Server with Script

This script can be run with or without any argument and provide the list of Job, item locked in DataStage environment by any user. This script will help release the locks if user doesn't locked the item.

Script Name : DSgetJobLocks.sh
Usage : ./DSgetJobLocks.sh [one or more argument]

Check whether DataStage Job is Multi-Instance or not with Sctipt

The following script can be used to check if the given datastage job is multi instance or not.

Arguments to the Scripts:

Arg1:Datastage Project Name
Arg2:JobName

Get DataStage Job Information without using Director

With the help of this script, You can get the job no, category and other information without opening DataStage Director.

This Script need two arguments :
1. DataStage Project Name
2. DataStage Job Name

Script :

DataStage Server Hang Issues & Resolution

Server hang issue can occurred when

1) Metadata repository database detects a deadlock condition and choose failing job as the victim of the deadlock.
2) Log maintenance is ignored.
3) Temp folders are not maintained periodically.

I will try to explain above three points in detail below:

What can you delete to free up disk space in IBM InfoSphere Information Server

What can you delete to free up disk space in IBM InfoSphere Information Server when disks are becoming full?

What to do ????

Here are some things you can do to clean up space:

• Clear the &PH& file in the Project directory. There is a &PH& directory in each DataStage project directory, which contains information about active stages that is used for diagnostic purposes. The &PH& directory is added to every time a job is run, and needs periodic cleaning out.
  

Deleting temporary lookuptable files in IBM InfoSphere DataStage

When a DataStage job with a lookup stage aborts, there may be lookuptable files left in the resource directories and they will consume space. The filenames are similar to "lookuptable.20091210.513biba"

When a job aborts it leaves the temporary files for postmortem review in the resource directories. Usually that is done in scratch, however, for lookup files, they are created in resource. Lookup filesets will not go away, just like regular datasets.

Managing and Deleting Persistent Data Sets within IBM InfoSphere Datastage

Data Sets sometimes take up too much disk space. This technote describes how to obtain information about datasets and how to delete them.

 

Data sets can be managed using the Data Set Management tool, invoked from the Tools > Data Set Management menu option within DataStage Designer (DataStage Manager in the 7.5 releases.) Alternatively, the 'orchadmin' command line program can be used to perform the same tasks.

The files which store the actual data persist in the locations identified as resource disks in the configuration files. These files are named according to the pattern below:

descriptor.user.host.ssss.pppp.nnnn.pid.time.index.random

Commands to delete files in &PH& directories in IBM InfoSphere DataStage

How should we manage cleaning up the &PH& directories for my DataStage projects? Can the process be automated?

Files in the &PH& directories under DataStage project directories store runtime information when jobs are run and need to be cleared out periodically.

 

Steps :

To clear the &PH& directory from within DataStage:

Error 39202 when logging into IBM DataStage

DataStage Client programs (Designer, Manager, Director) experience a connection failure, and the message displayed to the user is:

Failed to connect to host: xxxxx, project: UV
(Internal Error (39202))

DataStage Jobs hang or have slow performance when temp directories have many files

Datastage jobs write multiple files to temporary directories which are not automatically cleaned up. When the number of files grows large, DataStage jobs will have slower performance or can appear to hang.

DataStage jobs write multiple files to temporary directories which are not automatically cleaned up. When the number of files grows large, DataStage jobs will have slower performance or can appear to hang. For sites who have been running DataStage for a year or more and have not previously performed cleanup of these directories, they could contain 100,000 or more files.

Perform periodic cleanup of old DataStage files in these directories to prevent performance problems relating to the number of files in the directories.

When datastage file system becomes full, RT_LOGxxx files can become corrupted. Here is a manual procedure to recreate them

When many datastage jobs are running for long time, the RT_LOGxxx files can become very big. If the log files are not regularly purged, in some cases, a file system full on the datastage file system can occur. If the corruption of datastage system files is restricted to the RT_LOGxxx files, a procedure to delete and recreate the RT_LOGxxx files can be attempted in order to recover from the corruption.

Selecting DataStage job log results in error: Error Selecting from log file RT_LOG

When trying to view a job log in DataStage Director, an error similar to the following is received:

Error selecting from log file RT_LOGnn
Command was: SSELECT RT_LOGnn WITH @ID LIKE '1NON' COUNT.SU
Error was: Internal data error. File <path_to>/RT_LOGnn/<filename>':
Computed blink of 0xnnnn does not match expected blink of 0xnnnn! Detected within group starting at address 0xnnnnnn!
The error message received indicates that the log file for the job is corrupted.

DS Director Error (Unable to open Specific folder)

Reason -

Unable to open a specific folder in the director, it prompts the following error message. when I say, 'OK' , Director gets close. But there is no problem with the other folders. The Error message is "Run-time error '457'; This key is already associated with an element of this collection"

DataStage client receives error when compile jobs: Cannot get exclusive access to executable file for job sampleJob - job may be being monitored

Problem :

DataStage user sees the following error in Designer client when attempting to compile a specific job:
Cannot get exclusive access to executable file for job sampleJob - job may be being monitored.

Similar problems may occur in other releases.