ETL Job Design Standards - 2

Part 1 --> ETL Job Design Standards - 1

Parameter Management Standards

This section defines standards to manage job parameters across environments. Jobs should use parameters liberally to avoid hard coding as much as possible. Some categories of parameters include: 

• Environmental parameters, such as directory names, file names, etc.
• Database connection parameters
• Notification email addresses
• Processing options, such as degree of parallelism

Diagnosing DataStage Job Monitor problems

DataStage job run statistics (i.e. rows per second processed) do not update in DataStage Designer or Director clients.

 

Check these details :

This section contains a quick series of diagnosis steps for those familiar with DataStage and Job monitor. If more detail is needed for any step, please refer to the more detailed instructions in the "Resolving the Problem" section:
If customer has checked and verified that the following:

More Warnings causes Low Performance in DataStage. WHY ???

 
Warning messages are bad in any programming language; more so, in the case of DataStage. 

This is because DataStage is typically used for large scale processing of data. Every time a warning message is encountered, the engine code has to encounter an exception handler, throw the warning message and resume its task. 

Count User logins in Nix

This script will count the User logins on the Server and if you pass the username to script, It will fetch the last login details of the user.

Linux script to collect system statistics and send to your email

This script can use to daily send the system statistics server. It will also reformat the output to replace tabs with 5 spaces so it will display nicely on your email client.