DataStage Installation Checklist

• Review the system requirements.
• learn about tiers and components.
• Design the topology of your installation.
• Determine whether to pre-install your database system.
• Determine whether to pre-install IBM WebSphere Application Server or pre-configure an existing installation of WebSphere Application Server.

Data Warehouse Testing Checklist

Unit testing checklist

 A checklist will aid database programmers to systematically test their code before formal QA testing.

• Check the mapping of fields that support data staging and in data marts.
• Check for duplication of values generated using sequence generators.
• Check the correctness of surrogate keys that uniquely identify rows of data.
• Check for data-type constraints of the fields present in staging and core levels.
• Check the data loading status and error messages after ETLs (extracts, transformations, loads).
• Look for string columns that are incorrectly left or right trimmed.
• Make sure all tables and specified fields were loaded from source to staging.
• Verify that not-null fields were populated.
• Verify that no data truncation occurred in each field.
• Make sure data types and formats are as specified during database design.
• Make sure there are no duplicate records in target tables.
• Make sure data transformations are correctly based on business rules.
• Verify that numeric fields are populated precisely.
• Make sure every ETL session completed with only planned exceptions.
• Verify all data cleansing, transformation, and error and exception handling.
• Verify stored procedure calculations and data mappings.

Error 39202 when logging into IBM DataStage

DataStage Client programs (Designer, Manager, Director) experience a connection failure, and the message displayed to the user is:

Failed to connect to host: xxxxx, project: UV
(Internal Error (39202))

Orchadmin Command : DataStage

Orchadmin is a command line utility provided by datastage to research on data sets.

The general callable format is : $orchadmin <command> [options] [descriptor file]

1. Before using orchadmin, you should make sure that either the working directory or the $APT_ORCHHOME/etc  contains the file “config.apt” OR

The environment variable $APT_CONFIG_FILE  should be defined for your session.

Dropping CheckPoints in Job design

ETL tools can be extremely involved, especially with complex data sets. At one time or another, many data management professionals have built tools that have done the following:

• Taken data from multiple places.
• Transformed into (often significantly) into formats that other systems can accept.
• Loaded said data into new systems.

In this post, I discuss how to add some basic checkpoints into tools to prevent things from breaking bad.