Parallel jobs on Windows fail with APT_IOPort::readBlkVirt;error

APT_IOPort::readBlkVirt Error Screenshot

APT_IOPort::readBlkVirt Error Screenshot

This a known error for windows systems and applies to DataStage and DataQuality jobs using the any or all the three join type stages (Join, Merge, and Lookup).

Error Message

  • <<Link name>>,0: APT_IOPort::readBlkVirt: read for block header, partition 0, [fd 4], returned -1 with errno 10,054 (Unknown error)

Message ID

  • IIS-DSEE-TFIO-00223

Applies To

  • Windows systems only
  • Parallel Engine Jobs the three join type stages (Join, Merge, and Lookup). It does not apply to Server Engine jobs.
  • Infosphere Information Server (IIS), Datastage and DataQuality 9.1 and higher

The Fix

  • Add the APT_NO_IOCOMM_OPTIMIZATION in project administrator and set to blank or 0. I left it blank so it would not impact other jobs
  • Add the environment variable to the job producing the error and set to 1

What it APT_NO_IOCOMM_OPTIMIZATION Does

  • Sets the use of shared memory as the transport type, rather than using the default sockets transport type.
  • Note that in most cases sockets transport type is faster, so, you likely will not to set this across the project as the default for all job. It is best to apply it as necessary for problematic jobs.

Related References

InfoSphere DataStage and QualityStage, Version 9.1 Job Compatibility

IBM Support, JR54078: PARALLEL JOBS ON WINDOWS FAIL WITH APT_IOPORT::READBLKVIRT; ERROR

IBM Support, Information Server DataStage job fails with unknown error 10,054.

 

Aginity For Netezza – How to Generate DDL

Aginity, Aginity for Netezza, Netezza, PureData, DDL, SQL

Aginity

How to Generate Netezza Object DDL

In ‘Aginity for Netezza’ this process is easy, if you have a user with sufficient permissions.

The basic process is:

  • In the object browser, navigate to the Database
  • select the Object (e.g. table, view, stored procedure)
  • Right Click, Select ‘Script’ > ‘DDL to query window’
  • The Object DDL will appear in the query window
Create DDL to Query Window

Create DDL to Query Window

Related References

 

[HY000] ERROR: Base table/view has changed (datatype); rebuild view

Error

Error

Error Message

ERROR [HY000] ERROR: Base table/view ‘BLOG_LIST_TBL’ attr ‘BLOG_ID’ has changed (datatype); rebuild view ‘BLOG_LIST_VW’

What does the Error Mean?

This error means that an underlying table used by the view has changed in a way, which made the view invalid, as a result, the view must be rebuilt to reflect the new table definition.

How to Rebuild the view

In ‘Aginity for Netezza’ this process is easy, if you have a user with sufficient permissions.  The basic process is:

  • Navigate to and select the view to be rebuilt
  • Right Click, Select ‘Script’ > ‘DDL to query window’
Create DDL to Query Window

Create DDL to Query Window

  • Once the ‘Create or Replace View’ SQL has generated, click within the SQl statement (without highlighting)
  • Press Ctrl+F5 or navigate to Execute > ‘Execute as a Single batch
Execute SQL as a Single Batch

Execute SQL as a Single Batch

 

  • Verify the ‘Create or Replace View’ SQL executes successfully
  • Then, run a simple select against the view and that Select runs without producing the HYOOO Base Table/View Error

DataStage – IIS-DSEE-TBLD-00008- Processing Input Record APT_Decimal Error

IIS-DSEE-TBLD-00008 apt decimal error before disabling combination

IIS-DSEE-TBLD-00008 apt decimal error

This another one of those nebulas error messages, which can cost a lost of time in research, if you don’t know how to simplify the process a bit.  However, determining where the error is can be a bit of a challenge if you have not encountered this error before and figured out the trick, which isn’t exactly intuitive.

In this case, as it turned out, after I had determined where the error was, it was as simple as having missed resetting the stage variable properties, when the other decimal fields were increase.

How to identify where this error occurs?

Disabling the APT_DISABLE_COMBINATION environment variable by:

  • adding the APT_DISABLE_COMBINATION environment variable to the job properties

  • setting the  APT_DISABLE_COMBINATION environment variable it to true in the job properties

  • compiling the job and running the job again

 

This approach will, usually, provide a more meaningful identification of the stage with the error.

Note:  Please remember to remove the APT_DISABLE_COMBINATION environment variable before moving it to testing and/or releasing your code in production.

Message ID

IIS-DSEE-TBLD-00008

Error Message with combine enabled:

APT_CombinedOperatorController(1),0: Exception caught in processingInputRecord() for input “0”: APT_Decimal::ErrorBase: From: the source decimal has even precision, but non-zero in the leading nybble, or is too large for the destination decimal… Record dropped. Create a reject link to save the record if needed.

Error message with combine disabled

Tfm_Measures_calc,0: Exception caught in processingInputRecord() for input “0”: APT_Decimal::ErrorBase: From: the source decimal has even precision, but non-zero in the leading nybble or is too large for the destination decimal… Record dropped. Create a reject link to save the record if needed.

IIS-DSEE-TBLD-00008 apt decimal error after disabling combination

IIS-DSEE-TBLD-00008 apt decimal error after disabling combination

Note: Measures_calc is the stage name

Related References

 

 

Infosphere – decimal_from_string Conversion Error

IBM Infosphere - decimal_from_string Conversion Error

decimal_from_string Conversion Error

This is another one of those nebulas error, which can kick out of DataStage, DataQuality, and/or DataClick.  This error can be particularly annoying, because it doesn’t identify the field or even the precise command, which is causing the error.  So, there can be more than field and/or more than one command causing the problem.

Error

Conversion error calling conversion routine decimal_from_string data may have been lost

Resolution

To resolve this error, check for the correct formatting (date format, decimal, and null value handling) before passing to datastage StringToDate, DateToString,DecimalToString or StringToDecimal functions.  Additionally, even if the formatting is correct, you may need to imbed commands to completely clear the issue.

Example

Here is a recent example of command embedding, which has clear the issue, but I’m sure you will need to this concept in other ways to meet all your needs.

DecimalToString( DecimalToDecimal( <>,’trunc_zero’),”suppress_zero”)

How to suppress a Change_Capture_Host warning in Datastage

Change Capture Host Warning (IIS-DSEE-TFXR-00017)

Change Capture Host Warning (IIS-DSEE-TFXR-00017)

Occasionally, I run into this Change Capture Host defaulting warming, so, I thought this information may be useful.

Event Type

  • Warning

Message ID

  • IIS-DSEE-TFXR-00017

Example Message

  • Change_Capture_Host: When checking operator: Defaulting “<<FieldName>>” in transfer from “beforeRec” to “outputRec”.

Setting Variable

  • Set APT_DISABLE_TFXR0017=1
  • This environment variable can be added either at the project level or at the job level.

Alternative Solution

  • Within the Change Capture stage properties:
    • Stage tab
    • Option
    • Property: “Change Mode
    • Value:  “Explicit key, All Values”.

What does “ERROR: pg_atoi: error in : can’t parse ” mean?

This is one of those vague messages, which takes a little time to run down, but once you understand it, is easily fixed.  Basically, this error happens because you have mismatch and incompatible fields.

Question

What does “ERROR: pg_atoi: error in <ascii>: can’t parse <ascii>” mean?

Answer

The error is the result of an implicit conversion being performed on a character field (varchar/char) while being compared to a numeric field or value.

Vendor Reference Link

Infosphere Data Architect Install Error CRIMC1029E or CRIMC1085E

The CRIMC1029E / CRIMC1085E errors may be caused by running the incorrect Infosphere Data Architect installer.  If you run the admin installer (launchpad.exe) on 64bit windows with insufficient privileges, the process will throw a CRIMC1029E / CRIMC1085E error.

What the Error Looks Like

 

Installation failed.

CRIMC1029E: Adding plug-in com.ibm.etools.cobol.win32_7.0.921.v20140409_0421 to repository E:\Program Files\IBM\SDPShared failed.

CRIMC1085E: Resumable download failed for: file:/E:/InstallIBM/INFO_DATA_ARCH_V9.1.2_WIN/disk2/ad/plugins/file000086.

‘plug-in com.ibm.etools.cobol.win32_7.0.921.v20140409_0421’ does not exist (at file:/E:/InstallIBM/INFO_DATA_ARCH_V9.1.2_WIN/disk2/ad/plugins/file000086).

Elapsed time 00:00.00.

Solution

Run the launchpad_win_nonadmin64.exe file, instead of the launchpad.exe file.

Reference Link:

Installing IBM InfoSphere Data Architect with the Installation Manager Install wizard

 

What are the Infosphere DataStage job status log values?

IBM Infosphere Information Server (IIS), DataStage job status log values, datastage

IBM Infosphere Information Server (IIS)

These are the job status codes seen when running Infosphere Datastage jobs and sequences.   Additional information regarding a specific job or sequence error can be seen in Director.

Table of dsjob utility Status Codes

dsjob utility Status Codes

 

Log/Status Description Job State Comments
0 Running Not Runnable This is the only status that means the job is actually running
1 Finished Runnable Job finished a normal run with no warnings
2 Finished (See Log) Runnable Job finished a normal run with warnings
3 Aborted Not Runnable Job finished a normal run with a fatal error
4 Queued Not Runnable Job queued waiting for resource allocation
8 Failed Validation Not Runnable
9 Has Been Reset Runnable
11 Validated OK Runnable
12 Validated (See Log) Runnable
13 Failed Validation Not Runnable
21 Has Been Reset Runnable
96 Aborted Not Runnable
97 Stopped Not Runnable
98 Not Compiled Not Runnable