tag:blogger.com,1999:blog-79914907335649053682024-02-19T18:35:24.045+05:30Nuts & Bolts of DataStageSomething about DataStage, DataStage Administration, Job Designing,Developing, DataStage troubleshooting, DataStage Installation & Configuration, ETL, DataWareHousing, DB2, Teradata, Oracle and Scripting.Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.comBlogger513125tag:blogger.com,1999:blog-7991490733564905368.post-13204770777110582812015-10-01T15:03:00.000+05:302015-10-01T15:03:21.942+05:30My Blog Report - till now<br />
First of all, I would like to say thank you all for loving this blog so much. As you are aware we have moved to<a href="http://www.datagenx.net/" target="_blank"> <b>"www.DataGenX.net"</b></a> now. Keep Learning, Keep Sharing n Keep loving us :-)<br />
<br />
Today I am sharing Blog Report for "Nuts & Bolts of DataStage".<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmePyaJa24srsmZLG1s8GRD198MS6HF_rTDJ3LlDj9DF-tm8HkHv_aU8Sf-imYCBDJ3yXQIyvZdVDrRjGrp7O5ahS6rFM8ZQk7e7-SgPqg4uz0gEBmI7k8aVYqC-nTlRKnmrwhk5lLl8w/s1600/Audience.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="592" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmePyaJa24srsmZLG1s8GRD198MS6HF_rTDJ3LlDj9DF-tm8HkHv_aU8Sf-imYCBDJ3yXQIyvZdVDrRjGrp7O5ahS6rFM8ZQk7e7-SgPqg4uz0gEBmI7k8aVYqC-nTlRKnmrwhk5lLl8w/s640/Audience.PNG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZaDrspuih6riqKeyMDgvunVxtkDieo7oPorjqgSl0Rxn5pg_zmniGQsh-dUb7mW_EOWxgiAgG2uHiYSjGlmdgbI43ylJX_lp7Y7lufud-P5TNns9QJnLNaLmkAW_ND4PJYPPOz9UKAPo/s1600/MostClicked.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="530" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZaDrspuih6riqKeyMDgvunVxtkDieo7oPorjqgSl0Rxn5pg_zmniGQsh-dUb7mW_EOWxgiAgG2uHiYSjGlmdgbI43ylJX_lp7Y7lufud-P5TNns9QJnLNaLmkAW_ND4PJYPPOz9UKAPo/s640/MostClicked.PNG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK9NWCFlU-zOra7t1q5gpxNtV01l01MVyc_wKkLcIWq8ulT1tl-2PUM4AUiYrNt4Zgs3sfhFtrHVOXKV-I8SyXvXopp1ClWg3VCdxjX40F5t8IqaVVUORvA9Came5VLsRDMm0p92b-NQg/s1600/traffic.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK9NWCFlU-zOra7t1q5gpxNtV01l01MVyc_wKkLcIWq8ulT1tl-2PUM4AUiYrNt4Zgs3sfhFtrHVOXKV-I8SyXvXopp1ClWg3VCdxjX40F5t8IqaVVUORvA9Came5VLsRDMm0p92b-NQg/s640/traffic.PNG" width="594" /></a></div>
<br />
<br />
<br />
<br />
<i> <b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/data</a>genx<br />
<a href="https://groups.google.com/d/forum/datagenx" target="_blank">https://groups.google.com/d/forum/d</a>atagenx<br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-90948252252549336182015-08-18T20:51:00.002+05:302015-08-24T19:38:59.813+05:30DataGenXHi Guys,<br />
<br />
We have moved to new Site now , Kindly Visit<br />
<b><span style="font-size: x-large;"><br /></span></b>
<b><span style="font-size: x-large;"><i><a href="http://www.datagenx.net/">http://www.datagenx.net/</a></i></span></b><br />
<br />
<br />
<h3>
Twitter -- <a href="https://twitter.com/datagenx">https://twitter.com/datagenx</a><br />Reddit -- <a href="https://www.reddit.com/user/datagenx">https://www.reddit.com/user/datagenx</a><br />Pinterest -- <a href="https://www.pinterest.com/atulsingh0/datagenx/">https://www.pinterest.com/atulsingh0/datagenx/</a></h3>
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-534757201440420072015-08-04T19:14:00.003+05:302015-08-04T19:14:53.359+05:30Regular Expressions in Linux<br />
<div class="MsoNoSpacing">
<b>What Are Regular
Expressions?<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
A regular expression is a pattern template you define
that a Linux utility Uses to filter text. A Linux utility (such as the sed
editor or the awk program) matches the regular expression pattern against data
as that data flows Into the utility. If the data matches the pattern, it's
accepted for processing.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
If
the data doesn't match the pattern, it's rejected. The regular expression
pattern makes use of wildcard characters to represent one or more characters in
the data stream.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Types of regular
expressions:<o:p></o:p></b></div>
<div class="MsoNoSpacing">
There are two popular regular expression engines:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The POSIX Basic Regular Expression (BRE) engine<o:p></o:p></div>
<div class="MsoNoSpacing">
The POSIX Extended Regular Expression (ERE) engine<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b style="text-indent: -0.25in;">A.<span style="font-size: 7pt; font-stretch: normal; font-weight: normal;"> </span></b><b style="text-indent: -0.25in;">Defining BRE Patterns:</b></div>
<div class="MsoNoSpacing" style="margin-left: .5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<o:p></o:p></div>
<div class="MsoNoSpacing">
The most basic BRE pattern is matching text characters in
a data stream.<o:p></o:p></div>
<div class="MsoNoSpacing">
<b></b></div>
<a name='more'></a><b>Example 1: Plain
text</b><br />
<div class="MsoNoSpacing">
<o:p> </o:p> </div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
is a test" | sed -n '/test/p'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">This is a test.<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
is a test" | sed -n '/trial/p'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
is a test" | awk '/test/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">This is a test.<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
is a test" | awk '/trial/{print $0}’<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 2: Special
characters<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<o:p> </o:p> </div>
<div class="MsoNoSpacing">
The special characters recognized by regular expressions
are:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>. * [ ] ^ $ { } \ + ? | ( )</b><o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
For example, if you want to search for a dollar sign in
your text, just precede it with a backslash character:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ cat data2<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">The cost is $4.00<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ sed -n '/\$/p'
data2<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">The cost is $4.00<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 3: Looking
for the ending<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<o:p> </o:p> </div>
<div class="MsoNoSpacing">
The dollar sign ($) special character defines the end
anchor.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
is a good book" | sed -n '/book$/p'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">This is a good
book<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "This
book is good" | sed -n '/book$/p'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 4: Using
ranges<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<o:p> </o:p> </div>
<div class="MsoNoSpacing">
You can use a range of characters within a character
class by using the dash symbol.<o:p></o:p></div>
<div class="MsoNoSpacing">
Now you can simplify the zip code example by specifying a
range of digits:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ sed -n
'/^[0-9][0-9][0-9][0-9][0-9]$/p' data8<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">60633<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">46201<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">45902<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b style="text-indent: -0.25in;">B.<span style="font-size: 7pt; font-stretch: normal; font-weight: normal;"> </span></b><b style="text-indent: -0.25in;">Extended Regular Expressions:</b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The POSIX ERE patterns include a few additional symbols
that are used by some Linux applications and utilities. The awk program
recognizes the ERE patterns, but the sed editor doesn't.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 1: The
question mark<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The question mark indicates that the preceding character
can appear zero or one time, but that's all. It doesn't match repeating
occurrences of the character:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"bt" | awk '/be?t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">bt<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"bet" | awk '/be?t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">Bet<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"beet" | awk '/be?t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"beeet" | awk '/be?t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 2: The
plus sign<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The plus sign indicates that the preceding character can
appear one ormore times, but must be present at least once. The pattern doesn't
match if the character is not present:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"beeet" | awk '/be+t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">beeet<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"beet" | awk '/be+t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">beet<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"bet" | awk '/be+t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">bet<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"bt" | awk '/be+t/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 3: The
pipe symbol<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The pipe symbol allows to you to specify two or more
patterns that the regular expression engine uses in a logical OR formula when
examining the data stream. If any of the patterns match the data stream text,
the text passes. If none of the patterns match, the data stream text fails.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
The format for using the pipe symbol is:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
expr1|expr2|...<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
Here's an example of this:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "The
cat is asleep" | awk '/cat|dog/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">The cat is asleep<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "The
dog is asleep" | awk '/cat|dog/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">The dog is asleep<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo "The
sheep is asleep" | awk '/cat|dog/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b>Example 4:
Grouping expressions<o:p></o:p></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
When you group a regular expression pattern, the group is
treated like a standard character. You can apply a special character to the
group just as you would to a regular character.<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
For example:<o:p></o:p></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"Sat" | awk '/Sat(urday)?/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">Sat<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$ echo
"Saturday" | awk '/Sat(urday)?/{print $0}'<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">Saturday<o:p></o:p></span></div>
<br />
<div class="MsoNoSpacing">
<span style="font-family: "Courier New";">$<o:p></o:p></span></div>
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-52745438468263966592015-07-15T17:04:00.002+05:302015-07-15T19:51:01.837+05:30Error while creating new jobs in DataStage<br />
If you face the error error similar to below can occur when trying to save a newly created job:<br />
<br />
<span style="color: #e06666;">Error On CREATE.FILE command</span><br />
<span style="color: #e06666;">Creating file "RT_CONFIG4817 as type 30, mkdbfile: connot create file</span><br />
<span style="color: #e06666;">RT_CONFIG4817. Unable to create operating system file "RT_CONFIG4817"</span><br />
<br />
<a name='more'></a><br />
<h3>
Solution :</h3>
<br />
This error usually comes when the maximum number of sub-directories is reached in the project directory.<br />
<br />
1. To identify if the limit of sub-directories in the project directory has been reached, run the command below. An error will be given in case maximum number has reached:<br />
<br />
<span style="color: purple;">mkdir Test</span><br />
<br />
If you are able to create means this is not the one which is causing the issue, continue with your analysis .<br />
<br />
2. If not, means the maximum number of sub-directories is reached.<br />
<br />
<br />
<h3>
To Solve this issue we can follow below steps :-</h3>
<br />
Exporting jobs in the project category by category, reallocate jobs into new projects;<br />
or<br />
Move some sub-directories to another directory then use a softlink to link them back into the project directory. This approach requires lots of system management efforts and is recommended to engage with your systems administrator.<br />
<br />
I will prefer the first one.<br />
<br />
<div>
<br /></div>
<div>
<br /></div>
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-75725857043923671712015-07-13T15:32:00.000+05:302015-07-13T15:32:10.159+05:30Types of Jobs in DataStage<br />
<h3>
Server jobs</h3>
<br />
<ul>
<li>Executed by the DataStage server engine</li>
<li>Compiled into Basic</li>
<li>Runtime monitoring in DataStage Director</li>
<li>No parallel capability<a name='more'></a></li>
</ul>
<br />
<h3>
Parallel jobs</h3>
<br />
<ul>
<li>Executed by the DataStage parallel engine</li>
<li>Built-in functionality for pipeline and partition parallelism</li>
<li>Compiled into OSH (Orchestrate Scripting Language)</li>
<li> OSH operators( Executable C++ class instances)</li>
</ul>
<br />
<h3>
Mainframe jobs</h3>
<br />
<ul>
<li>Generate jobs that are compiled and run on a mainframe computer</li>
</ul>
<br />
<h3>
Job sequences (batch jobs, controlling jobs)</h3>
<br />
<ul>
<li>Master Server jobs that kick-off jobs and other activities</li>
<li>Can kick-off Server or Parallel jobs</li>
<li>Runtime monitoring in DataStage Director</li>
<li>Executed by the Server engine </li>
</ul>
<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-19715634974828868852015-07-08T19:23:00.002+05:302015-07-08T19:23:48.273+05:30What is EPOCH ?<br />
EPOCH is time in seconds since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970 ( 1970/01/01 00:00:00 UTC). Or the epoch is Unix time 0 (midnight 1/1/1970), but 'epoch' is often used as a synonym for 'Unix time'. EPOCH time is widely used in every field of IT sector (specially in unix like OS ) where we have to keep difference is generation of something based on time.<br />
<br />
<a name='more'></a><br />
These are the way to convert current time to EPOCH in different DBs or Languages -<br />
<br />
Python<span class="Apple-tab-span" style="white-space: pre;"> </span>import time first, then time.time()<br />
Java<span class="Apple-tab-span" style="white-space: pre;"> </span>long epoch = System.currentTimeMillis()/1000;<br />
C#<span class="Apple-tab-span" style="white-space: pre;"> </span>var epoch = (DateTime.UtcNow - new DateTime(1970, 1, 1)).TotalSeconds;<br />
Oracle PL/SQL<span class="Apple-tab-span" style="white-space: pre;"> </span>SELECT (SYSDATE - TO_DATE('01/01/1970 00:00:00', 'MM-DD-YYYY HH24:MI:SS')) * 24 * 60 * 60 FROM DUAL<br />
SQL Server<span class="Apple-tab-span" style="white-space: pre;"> </span>SELECT DATEDIFF(s, '1970-01-01 00:00:00', GETUTCDATE())<br />
Unix/Linux Shell<span class="Apple-tab-span" style="white-space: pre;"> </span>date +%s<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-4284448478687997742015-07-03T11:44:00.000+05:302015-07-03T11:45:53.953+05:30Get list of activated tools in Datastage Installation<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">If we want to check what features have been installed with datastage, we can check it by using <span class="keyword cmdname" style="background-color: white; font-weight: bold;">LicensingServiceAdmin</span><span style="background-color: white;"> tool. This will list down all the tool packs or features including Qualitystage as it comes as a part of datastage.</span></span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="background-color: white;"><br /></span>
</span><br />
<span style="background-color: white; font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Tool Location : {IS_INSTALL_DIR}/ASBServer/bin</span><br />
<a name='more'></a><br />
<br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Script Name : <span class="keyword cmdname" style="background-color: white; font-weight: bold;">LicensingServiceAdmin.sh</span></span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Simply execute below command -</span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<pre class="pre codeblock" style="background-color: #dadada; padding: 5px; white-space: pre-wrap; word-wrap: break-word;"><code><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">./LicensingServiceAdmin.sh -user <var class="keyword varname">iauser</var> -password <var class="keyword varname">iapswd</var> -list_features</span></code></pre>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">This will give output which features are activated (means you have installed license for them), Not activated features will not be displayed.</span><br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-47751493181769805662015-06-30T23:07:00.000+05:302015-06-30T23:07:21.796+05:30Maximum number of DataStage Jobs in a project<h2 style="background-color: white; border: 0px none; font-family: HelveticaNeue-Light, 'Helvetica Neue Light', 'Helvetica Neue', Helvetica, Arial; font-size: 1.166em; height: auto; margin: 5px 0px 0px; outline: none 0px; padding: 5px 0px; vertical-align: baseline;">
<span style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: 12.8000001907349px; line-height: 15.3600006103516px;"><br /></span></h2>
<h2 style="background-color: white; border: 0px none; font-family: HelveticaNeue-Light, 'Helvetica Neue Light', 'Helvetica Neue', Helvetica, Arial; font-size: 1.166em; height: auto; margin: 5px 0px 0px; outline: none 0px; padding: 5px 0px; vertical-align: baseline;">
<span style="background-color: white; color: #333333; font-family: Arial, sans-serif; font-size: small; font-weight: normal; line-height: 15.3600006103516px;">Filesystems can have a hard limit on number of subdirectories that can be contained in a directory, for example 32766 (32K -2) for Linux</span></h2>
<div class="wordwrap" style="background-color: white; border: 0px none; font-family: Arial, sans-serif; line-height: 15.3600006103516px; outline: none 0px; padding: 6px 0px; vertical-align: baseline; word-wrap: break-word;">
<div style="color: #333333;">
<span style="line-height: 15.3600006103516px;">For every job, DataStage will create hash files as subdirectories in the project directory. Here is a list of the subdirectories created for every job. In this list "nn" represents the number (or id) of the job:</span></div>
<div style="color: #333333;">
<br /></div>
<div style="color: #333333;">
DS_TEMPnn</div>
<div style="color: #333333;">
RT_CONFIGnn</div>
<div style="color: #333333;">
RT_STATUSnn</div>
<div style="color: #333333;">
RT_LOGnn</div>
<div style="color: #333333;">
RT_BPnn</div>
<div style="color: #333333;">
RT_BPnn.O</div>
<div style="color: #333333;">
RT_SCnn (only for parallel jobs)</div>
<div style="color: #333333;">
</div>
<a name='more'></a><div style="color: #333333;">
</div>
<div style="color: #333333;">
<span style="line-height: 15.3600006103516px;">The folders have been created for jobs -- </span></div>
<div style="color: #333333;">
Parallel Job - 07</div>
<div style="color: #333333;">
Server Job - 06</div>
<div style="color: #333333;">
<br /></div>
<div style="color: #333333;">
<br /></div>
<div style="color: #333333;">
<b>Our assumptions are below --</b></div>
<div style="color: #333333;">
<br /></div>
<span style="color: #660000;">Total sub directories can be created is - 32766</span><br />
<span style="color: #660000;">Project threshold - 80 </span><br />
<span style="color: #660000;">No of Parallel job can be creates is - 32766/7 - 80 ~ <b>4600</b></span><br />
<span style="line-height: 15.3600006103516px;"><span style="color: #660000;">No of Server job can be creates is - 32766/6 - 80 ~ <b>5380</b></span></span><br />
<div style="color: #333333;">
<br /></div>
<div style="color: #333333;">
where 80 is for overhead of project default directory. In practice, the number might be smaller due to subdirectories not counted above, such as directories created by QualityStage jobs, some project folders etc.</div>
<div style="color: #333333;">
<br /></div>
<div style="color: #333333;">
This is just approximated calculation, variables can vary system to system.</div>
</div>
<h2 style="background-color: white; border: 0px none; font-family: HelveticaNeue-Light, 'Helvetica Neue Light', 'Helvetica Neue', Helvetica, Arial; font-size: 1.166em; height: auto; margin: 5px 0px; outline: none 0px; padding: 5px 0px; vertical-align: baseline;">
<br /></h2>
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-14837363907098133022015-06-29T11:17:00.001+05:302015-06-29T11:17:16.421+05:30Get nth Highest or Lowest data from Table<br />
Here, we are considering a table student and we need to find out Nth highest and lowest marks of a student.<br />
<br />
<h3>
Nth Highest</h3>
<span style="color: #990000;">select * from student s1 where N-1 = ( select count(distinct(marks)) from student s2 where s1.marks > s2.marks)</span><br />
<br />
<a name='more'></a><br />
<br />
<h3>
Nth Lowest</h3>
<span style="color: #990000;">select * from student s1 where N-1 = ( select count(distinct(marks)) from student s2 where s1.marks < s2.marks)</span><br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-30579966141993241932015-06-27T16:04:00.003+05:302015-06-27T16:04:31.108+05:30SQL Scenario - 1You have a table like below :<br />
<br />
Name<span class="Apple-tab-span" style="white-space: pre;"> </span>Age<span class="Apple-tab-span" style="white-space: pre;"> </span>Gender<br />
Atul<span class="Apple-tab-span" style="white-space: pre;"> </span> 11<span class="Apple-tab-span" style="white-space: pre;"> </span>F<br />
Ram<span class="Apple-tab-span" style="white-space: pre;"> </span>10<span class="Apple-tab-span" style="white-space: pre;"> </span>F<br />
Disha<span class="Apple-tab-span" style="white-space: pre;"> </span> 6<span class="Apple-tab-span" style="white-space: pre;"> </span>M<br />
Pavan<span class="Apple-tab-span" style="white-space: pre;"> </span> 8<span class="Apple-tab-span" style="white-space: pre;"> </span>F<br />
Mani<span class="Apple-tab-span" style="white-space: pre;"> </span> 10<span class="Apple-tab-span" style="white-space: pre;"> </span>M<br />
Lovely<span class="Apple-tab-span" style="white-space: pre;"> </span>10<span class="Apple-tab-span" style="white-space: pre;"> </span>M<br />
Rani <span class="Apple-tab-span" style="white-space: pre;"> </span> 9<span class="Apple-tab-span" style="white-space: pre;"> </span>M<br />
<br />
<a name='more'></a><br />
<b>Write a single sql query statement to update Age with +1 and swap the gender ( M->F and F->M )</b><br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-56651966873025803062015-06-27T15:33:00.000+05:302017-08-30T07:43:54.744+05:30Unlocking XMETA locks from jobs<h2 style="background-color: white; border: 0px none; font-family: HelveticaNeue-Light, 'Helvetica Neue Light', 'Helvetica Neue', Helvetica, Arial; font-size: 1.166em; height: auto; margin: 5px 0px 0px; outline: none 0px; padding: 5px 0px; vertical-align: baseline;">
</h2>
<div class="wordwrap" style="background-color: white; border: 0px none; color: #333333; font-family: Arial, sans-serif; font-size: 12.8000001907349px; line-height: 15.3600006103516px; outline: none 0px; padding: 6px 0px; vertical-align: baseline; word-wrap: break-word;">
How can I unlock an XMETA locked job?<br />
<br />
The job cannot be exported and when attempting to open the job, the following error appears:<br />
<span style="color: #990000;"><br /></span>
<span style="color: #990000;"><b><i>Unable to open the Job</i></b></span><br />
<span style="color: #990000;"><b><i>The Job 'TestJob' is locked by user 'dsadm'</i></b></span><br />
<span style="color: #990000;"><b><i>(SessionID '8F1B0326-4F7E-45A8-B8EC-EAA8C4780528')</i></b></span><br />
<br />
Do you want to continue and open a read-only view of the Job?</div>
<div class="ibm-domino-rtf" style="background-color: white; border: 0px none; font-family: Arial, sans-serif; font-size: 12.8000001907349px; line-height: 15.3600006103516px; margin: 0px; outline: none 0px; padding: 0px; vertical-align: baseline;">
<div class="wordwrap" style="border: 0px none; color: #333333; font-size: 1em; outline: none 0px; padding: 6px 0px; vertical-align: baseline; word-wrap: break-word;">
This issue arises when a lock still remains in XMETA for the corresponding job. Information Server will lock the job to ensure that there is no indeterminancy regarding the editing and saving of a job.<br />
The most common causes of this is when a user has closed a client window unexpectedly (e.g. Force Terminate).</div>
</div>
<br />
<a name='more'></a><br />
<h3>
Solutions :</h3>
<br />
<h2 style="background-color: white; border: 0px none; font-family: HelveticaNeue-Light, 'Helvetica Neue Light', 'Helvetica Neue', Helvetica, Arial; font-size: 1.166em; height: auto; margin: 5px 0px; outline: none 0px; padding: 5px 0px; vertical-align: baseline;">
</h2>
To remove the lock on the job, the following steps need to be taken:<br />
Check that another user has not left the job open accidentally. The same error appears if another user has opened the same job in a Designer or Director client.<br />
<b><br /> Explore the following in Information Server web client (http://<server>:9080):</b><br />
1. Open Administration tab -> Session Management -> Active Sessions.<br />
2. Select the session with the same 'Session ID' as the message.<br />
3. Disconnect selected session.<br />
<br />
<b> Remove using cleanup_abandoned_locks.sh. This occurs if the session is no longer displayed but lock is remaining in the XMETALOCKINFO table in XMETA repository. To perform this:</b><br />
1. Login to domain layer using either root or Administrator user (for Windows)<br />
2. cd /opt/IBM/InformationServer/ASBServer/bin (C:\IBM\InformationServer\ASBServer\bin for Windowss)<br />
3. ./cleanup_abandoned_locks.sh -U <user> -P <password> (cleanup_abandoned_locks.bat -U <user> -P <password> for Windows)<br />
<br />
<br />
In this case, select a user with Suite Administrator privileges.<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-611962748361470122015-06-24T19:27:00.000+05:302015-06-24T19:27:10.582+05:30What happen when Import Sequential File with Extra Column<br />
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
Job reads sequential file and load into RDBMS table.</div>
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
<br /></div>
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
What happen if sequential file has extra column in the back of each record?</div>
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
<br /></div>
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
It depends on data type of last column defined by DS and RDBMS table.</div>
<div style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">
<ul style="line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em;">
<li style="border: none; margin: 0px 0px 0.25em; padding: 0px;">If data type of last column = varchar, extra column will be included as last column.<a name='more'></a></li>
<li style="border: none; margin: 0px 0px 0.25em; padding: 0px;">If data type of last column = numeric, last column value will become 0 because there is delimiter character which usually not a numeric.</li>
<li style="border: none; margin: 0px 0px 0.25em; padding: 0px;">If data type of last column = date, extra column will be dropped.</li>
</ul>
</div>
<span style="background-color: white; color: #4d4d4d; font-family: tahoma, sans-serif; font-size: 11.8800001144409px; line-height: 16.6319999694824px;">Their result base on transformer stage behavior on implicit data type conversion.</span><br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-69095669484725528962015-06-21T17:49:00.000+05:302015-06-21T17:49:00.319+05:30DataStage Installation Checklist<div>
<div>
<br /></div>
<div>
<ul>
<li><span style="font-family: Trebuchet MS, sans-serif;">Review the system requirements.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">learn about tiers and components.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Design the topology of your installation.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Determine whether to pre-install your database system.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Determine whether to pre-install IBM WebSphere Application Server or pre-configure an existing installation of WebSphere Application Server.<a name='more'></a></span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Build and test the hardware infrastructure.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">On all tier computers, verify name resolution, and open TCP ports for InfoSphere Information Server across any applicable firewalls.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Update the kernel parameters and user limits on each computer.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Prepare each computer to run the InfoSphere Information Server installation program, and back up any critical data on the computers.</span></li>
<li><span style="font-family: Trebuchet MS, sans-serif;">Run the ISA Lite to check whether your system is ready for installation or not.</span></li>
</ul>
</div>
</div>
<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-44009600460549433952015-06-16T15:53:00.000+05:302015-06-29T11:48:24.480+05:30trim leading 0's from decimal in DataStage<br />
1. Convert the decimal (or whatever it was) to String.<br />
2. then trim '0's from left.<br />
<br />
<a name='more'></a><br />
<br />
<pre class="pre codeblock" style="background-color: #dadada; font-family: "Courier New"; font-size: 10pt; padding: 5px;"><code>Trim(DecimalTostring(In.Col),'0','L')</code></pre>
<br />
** These functions are available in transformer stage.<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.comtag:blogger.com,1999:blog-7991490733564905368.post-76333160899553694182015-06-05T14:22:00.004+05:302018-01-05T09:08:39.365+05:30Installing PyMongo with MongoDB in Linux<br />
1. Download <b>Pymongo Source</b> from <b><a href="https://pypi.python.org/pypi/pymongo/" target="_blank">HERE</a></b>.<br />
2. Then follow below steps --<br />
<a name='more'></a><br />
<script src="https://gist.github.com/atulsingh0/a01f55033fd18605f654.js"></script>
Now, you can use step #b and #c to complete the PyMongo installation.<br />
<br />
<br />
# Download SetupTools Source from <a href="https://pypi.python.org/pypi/setuptools" target="_blank"><b>HERE</b></a><br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-89779566592779166862015-06-04T14:03:00.000+05:302015-06-04T14:03:35.004+05:30Error - mongod shell doesn't start, /data/db doesn't exsist<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br />
<i><span style="color: purple;"><b>Error - mongod shell doesn't start, /data/db doesn't exsist</b></span></i><br />
<br />
<br />
If you face this error while starting MongoDB (mongod), most probably there are 2 reasons for this -<br />
<br />
<i>a. /data/db is not exist at all</i><br />
<i>b. mongod process don't have access on /data/db</i><br />
<br />
<h3>
Solution :-</h3>
<br />
The default db path for mongodb is /data/db/.<br />
So when you run:<br />
<a name='more'></a><br />
<pre class="pre codeblock" style="background-color: #dadada; font-family: 'Courier New'; font-size: 10pt; padding: 5px;"><code>
mongod
</code>
</pre>
<br />
<br />
MongoDB checks if /data/db/ is present and if the user has access to it. In your case, there is no such directory and hence the error. So resolution of this error is -<br />
<br />
Create /data/db for mongoDB and give appropriate access :<br />
<pre class="pre codeblock" style="background-color: #dadada; font-family: 'Courier New'; font-size: 10pt; padding: 5px;"><code>
$ mkdir -p /data/db
$ chmod -R 775 /data/db
# if you concerned about the db security then -
$ chown -R user:group /data/db # user/group - mongodb user/group which you want to give access
</code>
</pre>
<br />
<br />
If you dont want to use default db ( /data/db ), create your own db and pass the path with mongod command : -<br />
<pre class="pre codeblock" style="background-color: #dadada; font-family: 'Courier New'; font-size: 10pt; padding: 5px;"><code>
$ mkdir -p /app/MongoDataBase
$ chmod -R 775 /app/MongoDataBase
# if you concerned about the db security then -
$ chown -R user:group /app/MongoDataBase # user/group - mongodb user/group which you want to give access
</code>
</pre>
<br />
<br />
Now start the MongoDB with below command :<br />
<pre class="pre codeblock" style="background-color: #dadada; font-family: 'Courier New'; font-size: 10pt; padding: 5px;"><code>
mongod --dbpath /app/MongoDataBase
</code>
</pre>
<br />
<br /></div>
<div class="MsoNormal">
<br /></div>
</div>
<div class="MsoNormal">
<i><b>Like the Facebook Page & join Group</b></i></div>
<i><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-80254092143044088592015-06-03T17:00:00.001+05:302015-06-03T17:13:43.426+05:30MongoDB Configuration in Linux<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="background-color: white;"><i style="font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18.2000007629395px;"><b><a href="http://www.mongodb.org/downloads" style="text-decoration: none;" target="_blank">DOWNLOAD</a></b></i><span style="font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18.2000007629395px;"> the stable version of MongoDB. It will a tar file.</span></span><br />
<span style="font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Calibri, sans-serif; font-size: 10pt;">1. Create a folder
named 'learn' ( or whatever you want to give )<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt;">
<span style="font-family: Calibri, sans-serif; font-size: 10pt;"> </span><span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;">$ mkdir -p /learn/mongodb /app/dbMongo<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"> #
/learn/mongodb = Holds the MongoDB
source<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"> #
/app/dbMongo = Holds the MongoDB database<o:p></o:p></span><br />
<span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"><br /></span>
<span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiafsN3dVKaADpVeIwSifEcVsAG7Dj0E-DIZJqhLK9oo8buahPGcwbI3eJpBrYrZIToOhQmqEV8b9_N8qMzfZqRrOGadWjRGWBE58a5xZDao6pyAo6d6lXyoGBwPR6-VkNzb6mOdC2wtL4/s1600/Screenshot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="161" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiafsN3dVKaADpVeIwSifEcVsAG7Dj0E-DIZJqhLK9oo8buahPGcwbI3eJpBrYrZIToOhQmqEV8b9_N8qMzfZqRrOGadWjRGWBE58a5xZDao6pyAo6d6lXyoGBwPR6-VkNzb6mOdC2wtL4/s320/Screenshot.png" width="320" /></a></div>
<span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"><br /></span>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiX1oro5JxVCc8PWCK2K103qtGnWzioEi59hX-qiz8lJkrf8CAmqWTV-tTf0PfLmPC-d6excxkKPGvYizYTN2i_W8ogf9FEGCnnVdfL_9QMjsTlrGn5B-zgk7GpYb2PdYHszLtMztLi9c0/s1600/Screenshot-1.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="145" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiX1oro5JxVCc8PWCK2K103qtGnWzioEi59hX-qiz8lJkrf8CAmqWTV-tTf0PfLmPC-d6excxkKPGvYizYTN2i_W8ogf9FEGCnnVdfL_9QMjsTlrGn5B-zgk7GpYb2PdYHszLtMztLi9c0/s320/Screenshot-1.png" width="320" /></a></div>
<div style="margin: 0in 0in 0.0001pt;">
<span style="font-family: Calibri, sans-serif; font-size: 10pt;">2. Extract the Mongo DB zip file in /learn folder</span></div>
<div style="margin: 0in 0in 0.0001pt;">
<span style="font-family: 'Courier New'; font-size: 10pt;"> </span><span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;">$ tar -xvf
MongoDB.tar /learn/mongodb</span></div>
<div style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div style="margin: 0in 0in 0.0001pt;">
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span><span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;">3. change the permission of folder to user
who run the db here- In my case User - atul and Group
- atul<br />
</span><span style="background: white; color: #351c75; font-family: "Courier New"; font-size: 10.0pt;"> </span><span style="background: white; color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;">$ chown -R atul:atul /learn/mongodb
</span><span style="color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;">/app/dbMongo</span><span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br />
<br /><br />
</span><br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><br /></span>
<br />
<a name='more'></a><br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;">4. Create a configuration file for MongoDB and customize as per your
installation (if different) and place
the file in /app/ directory. For my
installation – <o:p><script src="https://gist.github.com/datastage4you/47c4f92def87959d45fc.js"></script></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
</div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<br /></div>
<br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;">5. Add these line in .bashrc file<br />
</span><span style="background: white; color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;"># User specific aliases and functions<br />
<br />
#Home location for MongoDB<br />
MONGO_HOME='/learn/mongodb'<br />
<br />
#Mongo DB start command alias<br />
alias SdbMongo='$MONGO_HOME/bin/mongod --config /app/dbMongo.conf' <br />
<br />
#Add MongoDB in PATH<br />
export PATH=$PATH:$MONGO_HOME/bin<o:p></o:p></span><br />
<br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;">6. for Starting MongoDB, simply execute the alias
which we have created.<br />
</span><span style="background: white; color: #0070c0; font-family: "Courier New"; font-size: 10.0pt;">$ SdbMongo</span><span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt;"><o:p></o:p></span><br />
<span style="color: #0070c0; font-family: Courier New;"><span style="background-color: white; font-size: 13.3333330154419px;"># or execute the below command</span></span><br />
<span style="color: #0070c0; font-family: Courier New;"><span style="background-color: white; font-size: 13.3333330154419px;">$ </span></span><span style="background-color: white; color: #0070c0; font-family: 'Courier New'; font-size: 13.3333330154419px;">$MONGO_HOME/bin/mongod --config /app/dbMongo.conf</span><br />
<br />
<br />
<span style="background: white; font-family: Calibri, sans-serif; font-size: 10pt; line-height: 115%;"><b>mongod.pid</b> file holds the
process id for mongoDB<br /><b>mongod.log</b> file hold the log of mongoDB</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibryi3GhyphenhyphencmwmWUFxW58dLSGofUU_yEt_R-U-74uwadFw-jtV3K0jj3Rvbjiv9IkwQGHksLWgxHPlj2dEgRVWU-gl7rEjhkIBzFHJ28p5b_1jBwlrXAT6CYbfWFiwfhmZiqUFwEXWWLEs/s1600/Screenshot-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibryi3GhyphenhyphencmwmWUFxW58dLSGofUU_yEt_R-U-74uwadFw-jtV3K0jj3Rvbjiv9IkwQGHksLWgxHPlj2dEgRVWU-gl7rEjhkIBzFHJ28p5b_1jBwlrXAT6CYbfWFiwfhmZiqUFwEXWWLEs/s640/Screenshot-2.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com3tag:blogger.com,1999:blog-7991490733564905368.post-4732808596223394852015-06-01T16:20:00.002+05:302015-06-01T16:20:29.354+05:30Trigger types in Sequencer Job<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: inherit;"><br /></span>
<br />
<div class="shortdesc" style="background-color: white;">
<span style="font-family: inherit;">While designing Sequencer jobs when we are using any Sequencer Stage ( Job activity, Command activity or others ), it has a tab named 'trigger' which actually determine the actions that occur when the activity runs. Each activity can output different trigger types. </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Three types of triggers are available, with some types having subtypes.</span></div>
<dl class="dl" style="background-color: white;">
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;"><span style="font-family: inherit;">Conditional</span></dt>
<dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;">A conditional trigger runs the target activity if the source activity fulfills the specified condition. The condition is defined by an expression, and can be one of the following types:</span><br />
<div class="tablenoborder" style="margin-bottom: 1em; margin-top: 1em;">
<a name='more'></a><table border="1" cellpadding="4" cellspacing="0" class="table" frame="border" rules="all" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border-collapse: collapse; border-color: black; color: black; margin: 0.5em 0em; width: 850px;" summary=""><thead align="left" class="thead">
<tr class="row" valign="bottom"><th class="entry" id="d95213e25" style="background-color: #dadada; padding: 0.1em 1em 0.3em;" valign="bottom"><span style="font-family: inherit;">Conditional trigger</span></th><th class="entry" id="d95213e27" style="background-color: #dadada; padding: 0.1em 1em 0.3em;" valign="bottom"><span style="font-family: inherit;">Action</span></th></tr>
</thead><tbody class="tbody">
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">OK</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Runs the target activity if the source activity succeeds</span></td></tr>
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Failed</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Runs the target activity if the source activity fails</span></td></tr>
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">ReturnValue</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Returns a routine or command when the activity runs</span></td></tr>
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Warnings</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Returns a message if the activity produces warnings</span></td></tr>
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Custom</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Runs a customized trigger for the activity, which you define as an expression</span></td></tr>
<tr class="row"><td class="entry" headers="d95213e25 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">User status</span></td><td class="entry" headers="d95213e27 " style="padding: 0.1em 1em 0.3em;" valign="top"><span style="font-family: inherit;">Returns a customized status message and write it to the log</span></td></tr>
</tbody></table>
</div>
</dd>
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;"><span style="font-family: inherit;">Unconditional</span></dt>
<dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;">An unconditional trigger runs the target activity once the source activity completes, regardless of what other triggers are run from the same activity.</span></dd>
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;"><span style="font-family: inherit;">Otherwise</span></dt>
<dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;">An otherwise trigger is used as a default where a source activity has multiple output triggers, but none of the conditional triggers ran</span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><br /></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><br /></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;">There are few things we should remember with respect to triggers on sequences and they are as follows:</span></span><br />
<span style="font-family: inherit;"><br style="color: #222222; line-height: 20px;" /><span style="color: #222222; line-height: 20px;">1 - Job activities in a sequence can have one input trigger and multiple output triggers.</span></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;">2 - Trigger name must be unique in the activity, which help us to debug in case of failure, however you can have multiple same named triggers in a sequence.</span></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;">3 - Trigger would fire if the source meets the conditions defined in the activity. </span></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;">4 - DataStage macros can be used in triggers.</span></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;"><br /></span></span></dd><dd class="dd" style="margin-left: 1em;"><span style="font-family: inherit;"><span style="color: #222222; line-height: 20px;">We can say, by using correct triggers we can control Run flow in sequencer depending on business case to case.</span></span></dd></dl>
</div>
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: xx-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />Datagenxhttp://www.blogger.com/profile/15060885508064326572noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-80780961254527808432015-05-30T10:17:00.001+05:302015-05-30T10:17:52.070+05:30Data Warehouse Load Methods <br />
A data warehouse is a warehouse that contains data. Sounds funny
doesn't it? Imagine a warehouse as a storage structure that may be
physical or logical. In the DW it's both...logical as in the data model representation and physical as to the hard disks and other computer
equipment that support the data warehouse.<br />
<br />
The first question
should be "How does the data get there?" Here are three key areas to keep in mind....<br />
<br />
<b>Availability </b>- will someone create a report with transactions completed 20 minutes ago or will it be with yesterday's data?<br />
<br />
<a name='more'></a><br />
<b>ETL Process</b> - does our ETL process handle constant loads or is it a once a day process?<br />
<br />
<b>Hardware
limitations</b> - do we have the hardware resources to allow people to
access data and create reports while we are inserting/updating data?<br />
<br />
<b>Types of ETL Processing:</b><br />
<br />
<b>1.
Batch load - </b>load all data once a day. At the company I work with we
do a batch load process that begins at midnight and hopefully a majority
is complete by 7 am. <i>(pretty common)</i><br />
<br />
<b>2. Trickle feed -</b> rows are loaded in the EDW as they are received in the collector <i>(the hardest to implement)</i><br />
<br />
<b>3.
Microbatch -</b> small batches of data are processed on either a time
threshold (every 2 hours) or a row threshold <i>(20,000 records in the
collector).</i><br />
<br />
<br /><i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: xx-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.comtag:blogger.com,1999:blog-7991490733564905368.post-84628962727093091472015-05-27T22:26:00.000+05:302015-05-27T22:26:22.124+05:30Data sources in DataStage<br />
IIS Datastage connectivity options give us a wide scope to connect with different source or targets. It's support RDBMS, ERP, z/OS DB, OLAP system and many more. <br />
<br />
Below listed Data Sources are available in IIS v11.3<br />
<br />
<br />
<a name='more'></a><br />
<br />
<table border="0" cellpadding="0" cellspacing="0" frame="border" rules="all" style="width: 581px;" summary=""><colgroup><col style="mso-width-alt: 12690; mso-width-source: userset; width: 260pt;" width="347"></col>
<col style="mso-width-alt: 8557; mso-width-source: userset; width: 176pt;" width="234"></col>
</colgroup><tbody>
<tr height="17" style="height: 12.75pt;">
<td class="xl67" height="17" id="d68e57" style="height: 12.75pt; text-align: center; width: 260pt;" width="347"><span style="color: white;"><span style="background-color: #351c75;"><b> Data source </b></span></span></td>
<td class="xl67" id="d68e59" style="border-left: none; width: 176pt;" width="234"><span style="color: white;"><span style="background-color: #351c75;"><b> Type
of data source </b></span></span></td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Amazon S3</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Big data files</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Cognos TM1</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Complex flat files</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Data sets<span style="mso-spacerun: yes;"> </span></td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Distributed
transactions</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Real-time
processing data source</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">External sources</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">External targets</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Files</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">File sets</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Greenplum</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Hierarchical data</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Real-time
processing data source</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM DB2®</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM Informix®</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM InfoSphere Change
Data Capture</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Real-time
processing data source</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM InfoSphere Classic
Federation Server for z/OS</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM InfoSphere Master
Data Management</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Enterprise
application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM Red Brick
Warehouse</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM UniVerse and
UniData</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM WebSphere ILOG
JRules (IBM Operational Decision Manager)</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Business
and analytic application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">IBM WebSphere MQ</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Real-time
processing data source</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">InfoSphere Streams</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Business
and analytic application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">iWay</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Java code</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Custom
connection</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">JDBC data sources</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Multiple
data sources</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Lookup file sets</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Microsoft SQL Server
and MS OLE<span style="mso-spacerun: yes;"> </span></td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Netezza</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">ODBC data sources</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Multiple
data sources</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Oracle</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Salesforce.com</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Enterprise
application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">SAP applications</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Enterprise
application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">SAP BW</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Enterprise
application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Sequential files</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Siebel</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Enterprise
application</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Stored procedures</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Multiple
data sources</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Sybase</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Teradata</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Database</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Unstructured data</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">File</td>
</tr>
<tr height="17" style="height: 12.75pt;">
<td headers="d68e57 " height="17" style="height: 12.75pt;">Various relational
database systems (DRS connector)</td>
<td class="xl68" headers="d68e59 " style="border-top: none; width: 176pt;" width="234">Multiple
data sources</td>
</tr>
</tbody></table>
<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-40808186524859421372015-05-25T12:36:00.001+05:302015-05-25T12:39:05.293+05:30Execution Steps in Transformer Stage - Explanation<br />
<div class="post-title entry-title" itemprop="name">
<span style="font-size: small;"><b> You can access Part1 Here - <span style="color: red;"><i>
</i></span></b></span><a href="http://datastage4you.blogspot.in/2015/05/execution-steps-in-transformer-stage.html"><span style="font-size: small;"><b><span style="color: red;"><i>Execution Steps in Transformer Stage</i></span> </b></span></a></div>
<div class="p" id="wp800757__wp815039" style="margin-top: 1em;">
Certain constructs are inefficient if they are included in output column
derivations, because they are evaluated once for every output column
that uses them. The following examples describe these constructs:</div>
<dl class="dl">
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;">The same part of an expression is used in multiple column derivations.</dt>
<dd class="dd" style="margin-left: 1em;">For example, if you want to use
the same substring of an input column in multiple columns in output
links, you might use the following test in a number of output columns
derivations:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>IF (DSLINK1.col1[1,3] = "001") THEN ...</code></pre>
In this case, the evaluation of the substring of DSLINK1.col1[1,3] is
repeated for each column that uses it. The evaluation can be made more
efficient by moving the substring calculation into a stage variable. The
substring is then evaluated once for every input row. This example has
thus stage variable definition for StageVar1:<br />
<a name='more'></a><br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>DSLINK1.col1[1,3]</code></pre>
Each column derivation starts with this test:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>IF (StageVar1 = "001") THEN ...</code></pre>
This example can be improved further by also moving the string
comparison into the stage variable. The stage variable is then defined
as follows:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>IF (DSLink1.col1[1,3] = "001") THEN 1 ELSE 0</code></pre>
Each column derivation starts with this test:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>IF (StageVar1) THEN</code></pre>
The improved construct reduces both substring function evaluations and string comparisons.</dd>
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;">An expression includes calculated constant values.</dt>
<dd class="dd" style="margin-left: 1em;">For example, a column definition might include a function call that returns a constant value:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>Str(" ",20)</code></pre>
This function returns a string of 20 spaces. The function is evaluated
every time the column derivation is evaluated. It is more efficient to
calculate the constant value once. You can assign an initial value to a
stage variable in the <span class="ph uicontrol">Variables</span> tab of the <span class="keyword wintitle">Stage Properties</span> window. The initial value is set to using this expression:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>Str(" ", 20)</code></pre>
You do not supply the derivation of the stage variable on the main
Transformer page. The initial value of the stage variable is evaluated
once, before any input rows are processed. Because the derivation
expression of the stage variable is empty, the stage variable is not
re-evaluated for each input row. Change any expression that previously
used the function<samp class="ph codeph" style="font-family: 'Courier New', Courier, monospace;"> Str(" ", 20)</samp> to use the stage variable instead.</dd><dd class="dd" style="margin-left: 1em;">The
same considerations apply to any expression, or part of an expression,
that generates a constant value. For example, the following expression
concatenates two strings:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>"abc" : "def"</code></pre>
The abcdef concatenation is repeated every time the column derivation is
evaluated. Since the subpart of the expression is constant, this
constant part of the expression can again be moved into a stage
variable, using the initial value setting to perform the concatenation
once.</dd>
<dt class="dt dlterm" style="font-weight: bold; margin-bottom: 0.5em; margin-top: 0.5em;">An expression requiring a type conversion is used as a constant, or it is used in multiple places</dt>
<dd class="dd" style="margin-left: 1em;">For example, an expression might include the following code:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>DSLink1.col1+"1"</code></pre>
In this example, the "1" is a string constant, and so must be converted
from a string to an integer each time the expression is evaluated. The
solution in this case is to change the constant from a string to an
integer:<br />
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>DSLink1.col1+1</code></pre>
If DSLINK1.col1 is a string field, however, then a conversion is
required every time the expression is evaluated. If an input column is
used in more than one expression, where it requires the same type
conversion in each expression, it is more efficient to use a stage
variable to perform the conversion once. You can create, for this
example, an integer stage variable, specify its derivation to be
DSLINK1.col1, and then use the stage variable in place of DSLink1.col1,
where that conversion is required. When you use stage variables to
evaluate parts of expressions, you must set the data type of the stage
variable correctly for that context. Otherwise, needless conversions are
required wherever that variable is used.</dd></dl>
The advice on the use of stage variables equally applies to loop
variables when you use the Transformer stage looping facilities. You add
the evaluation to a stage variable if it is evaluated once per input
row, or to a loop variable if it is evaluated once per looped output row
when a loop condition was specified.<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />
<div class="fb-comments" data-colorscheme="light" data-href="https://www.facebook.com/DataStage4you" data-numposts="1" data-width="850">
</div>
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-89792369791682663982015-05-21T18:18:00.001+05:302015-05-22T23:58:34.456+05:30Execution Steps in Transformer Stage<br />
I've been asked this questions so many times in interviews and by different practitioner also that What are the data processing steps when datastage is processing transformer, So here I tried to compiled. Have a look - <br />
<br />
<br />
To write efficient Transformer stage derivations, it helps to understand what items get evaluated and when.<br />
<br />
<a name='more'></a><div class="section">
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>Evaluate each stage variable initial value
For each input row to process:
Evaluate each stage variable derivation value, unless
the derivation is empty
For each output link:
Evaluate constraint, if true:
Evaluate each column derivation value
Write the output row
Next output link
Next input row</code></pre>
<div class="p" style="margin-top: 1em;">
<br />
The evaluation sequence for a Transformer stage that has a loop condition defined is:<br />
<br /></div>
<pre class="pre codeblock" style="background-color: #dadada; font-size: 12px; padding: 5px;"><code>Evaluate each stage variable initial value
For each input row to process:
Evaluate each stage variable derivation value (unless empty)
Evaluate each loop variable initial value
While the evaluated loop condition is true:
Evaluate each loop variable derivation value (unless empty)
For each output link:
Evaluate constraint, if true:
Evaluate each column derivation value
Write the output row
Next output link
Loop back to While
Next input row</code></pre>
<div class="p" id="wp800757__wp815036" style="margin-top: 1em;">
<br />
The stage variables, loop variables, and the columns within a link are evaluated in the order in which they are displayed on the parallel job canvas. Similarly, the output links are also evaluated in the order in which they are displayed.</div>
</div>
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br /><br />
<br />
<div class="fb-comments" data-href="https://www.facebook.com/DataStage4you" data-width="850" data-numposts="1" data-colorscheme="light"></div>
<br />
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-55090040145578021072015-05-20T15:30:00.002+05:302015-05-20T15:47:52.468+05:30Disable auto insertion of Partition and Sort<br />
Partitioner insertion and sort insertion each make writing a flow easier by alleviating the need for a user to think about either partitioning or sorting data. By examining the requirements of operators in the flow, the parallel engine can insert partitioners, collectors and sorts as necessary within a data flow.<br />
<br />
However, there are some situations where these features can be a avoided or not needed.<br />
If data is pre-partitioned and pre-sorted, and the InfoSphere DataStage job is unaware of this, you could disable automatic partitioning and sorting for the whole job by setting the following environment variables while the job runs:<br />
<a name='more'></a><br />
<ul>
<li> APT_NO_PART_INSERTION</li>
<li> APT_NO_SORT_INSERTION</li>
</ul>
<br />
You can set these as "TRUE" to disable auto insertion by datastage. You can also disable partitioning on a per-link basis within your job design by explicitly setting a partitioning method of Same on the Input pagePartitioning tab of the stage the link is input to.<br />
To disable sorting on a per-link basis, insert a Sort stage on the link, and set the Sort Key Mode option to Don't Sort (Previously Sorted).<br />
IBM advise that average users leave both partitioner insertion and sort insertion alone, let Datastage handle this and that power users perform careful analysis before changing these options. <br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-6476940775961555702015-05-19T19:19:00.001+05:302015-05-19T19:19:59.321+05:30SQL Assignment08 - Delete Statement<br />
1. Delete from customers.<br />2. Delete depositor of branches having number of customers between 1 and 3.<br />3. Delete branches havinng average deposit less than 5000.<br />4. Delete branches having maximum loan more than 5000.<br />5. Delete branches having deposit from Nagpur.<br />
<a name='more'></a>6. Delete deposit of Anil and Sunil if both are having same branch.<br />7. Delete deposit of Anil and Sunil if both are having living city Nagpur.<br />8. Delete deposit of Anil and Sunil if both are having same living city.<br />9. Delete deposit of Anil and Sunil if they are having less deposit than Vijay.<br />10. Delete deposit of Vijay.<br />11. Delete deposit of Ajay if Vijay is not a depositor.<br />12. Delete customer from BOMBAY city.<br />13. Delete depositors if the branch is VIRAR and depositor name is Ajay.<br />14. Delete depositors having deposit less than 500.<br />15. Delete borrower having loan more than 10000.<br />16. Delete borrower having loan more than 1000 and branch KAROLBAGH.<br />17. Delete the name of those depositor of VRCE branch who live in the city BOMBAY.<br />18. Delete borrower having branch name KAROLBAGH<br />19. Delete borrower of branches having average loan less than 1000.<br />20. Delete borrower of branches having the minimum number of customers <br /><br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0tag:blogger.com,1999:blog-7991490733564905368.post-3879674670574327652015-05-14T19:54:00.001+05:302015-05-15T18:53:19.363+05:30SQL Assignment07 - Update Statement<br />
1. Give 10% interest to all depositors.<br />
2. give 10% interest to all depositors having branch VRCE<br />
3. Give 10% interest to all depositors living in city Nagpur.<br />
4. Give 10% interest to all depositors having branch in city BOMBAY and living in Nagpur.<br />
5. Add 100 Rs to deposit of Anil and assign it to Sunil.<br />
6. Change the deposit of VRCE branch to 1000 and change the branch as VRCE_UPT<br />
7. Assign to the deposit of Anil the maximum deposit from VRCE branch<br />
<a name='more'></a>8. Change the living city of VRCE branch borrowers to Nagpur.<br />
9. Update deposit of Anil, give him max deposit from depositors in living city Nagpur.<br />
10. Deposit the sum of the deposits of Sunil and Vijay in the account of Anil.<br />
11. Transfer Rs 150 from Anil's account to Sunil.<br />
12. Transfer Rs 150 from Anil's account to Sunil onlyif both are having same branch.<br />
13. Transfer Rs 150 from Anil's account to Sunil onlyif both are living in Nagpur.<br />
14. Add Rs 100 to account of all those depositors who are having the highest deposit amount in their respective branches<br />
15. Add Rs 150 to the amount of all depositors having deposit higher than the average deposit<br />
16. Add Rs 180 to the amount of all depositors having deposit higher than the average deposit of their branch.<br />
17. Add Rs 130 to the amount of all the customers living in the city where their branch is located.<br />
18. Add Rs 170 to the amount of all depositors having deposits equal to or higher than the average deposit of their branch.<br />
<br />
<br />
<br />
<br />
<br />
<i>
<b>Like the Facebook Page & join Group</b><br /><a href="https://www.facebook.com/DataStage4you" target="_blank">https://www.facebook.com/DataStage4you</a><br /><a href="https://www.facebook.com/groups/DataStage4you" target="_blank">https://www.facebook.com/groups/DataStage4you</a></i><br />
<i><a href="https://twitter.com/datastage4you">https://twitter.com/datastage4you</a><br />
<a href="https://groups.google.com/d/forum/datastage4you">https://groups.google.com/d/forum/datastage4you</a><br /><span style="font-size: x-small;">For <b>WHATSAPP group</b> , drop a msg to <b>91-88-00-906098</b></span></i>
<br />
<br />Atul Singhhttp://www.blogger.com/profile/09643920613196486931noreply@blogger.com0