PI - Monitoring Stale Data

Monitoring and detecting "faulty" tags is easy and straight forward. But to monitor "stage" tags is not.

Tools to Monitor Stale Data

I. Stale and Bad Points tool

If you have not configured PI Interface monitoring (see following sections), consider using the PI System Management Tools "Stale and Bad Points" plug-in as part of a daily health check of the PI Server. Use this tool to find and view stale and bad points to help you diagnose the condition that produced the errors. For more information on using the Stale and Bad Points plug-in, consult the PI SMT Help file, available from the Help icon when running the plug-in or directly from the \PIPC\Help\PISMT.chm.

II. PIconfig Script added to daily backup

If you prefer a scripted approach similar to the Stale and Bad Points tool, consider adding a piconfig script to your daily backup routine. Here is an example where the script looks for tags that have not received data in the last 15 minutes (adjust that number for your data rates):

@table pisnap
@ostr tag,time
@select tag = * , time < '*-15m'
@output daily-stale-tags.txt
@ends
@exit

III. Interface I/O Rate Tag

A single Input/Output (I/O) Rate point can be configured via the PI Interface Configuration Utility (ICU) IO Rate page to receive 10-minute averages of the total number of exceptions per minute that are sent to PI by the interface. (An exception is a value that has passed the exception specifications for a given PI point.)

A significant drop in the I/O Rate tag value signifies that the points assigned to this interface have not been sending data as normally expected, which could indicate stale data. (Note: This I/O Rate tag is different than the Interface Health I/O Rate Tag discussed in the next section.)

IV. Interface Health Tags

UniInt-based interfaces support a set of health tags that allow a PI system administrator to monitor the performance of the interface. The interface health tags can be created manually in the Interface Configuration Utility (ICU) via the UniInt> Health Points menu. (For EA customers, Managed PI automatically creates these tags and monitors them via the NOC.) A few health tags that can assist in troubleshooting stale data are:

Heartbeat [UI_HEARTBEAT]
IO Rate (events per second) [UI_IORATE]
Trigger Input Rate [UI_TRIGGERRATE]
Scan Class IO Rate [UI_SCIORATE]
Scan Class Scans Skipped [UI_SCSKIPPED]
For more information on Interface Health Tags, see the "General Interface Health Tags" section of the Interface Configuration Utility User Manual.

V. Check Interface Buffering

Examining buffering performance can indicate whether or not data is stale in PI due to events being stuck in the buffer queues. A potential cause for this is network issues. After an ungraceful shutdown, buffer queues may occasionally become corrupt, which will prevent data from flowing to the PI Server.

If you are running PI Buffer Subsystem, you can view your buffered sources from the PI Server using piartool -bfs, and then identify which stale points are associated with the registered buffered sources (PI Interface nodes). For more information on examining buffered sources, see KB00127.

Once you've identified stale tags with a particular interface node, you can run tools on the interface node to check if events are indeed stuck in the buffer queues. Please reference KB00363 for more information on how to view buffering statistics with PI Buffer Subsystem or API Buffer Server.

VI. Calculations and Notifications

Other tools such as PI Performance Equations, PI Statistical Quality Control and PI Advanced Computing Engine (PI ACE) calculations can be utilized to watch for stale data. PI Notifications can be used to notify users of stale tag events, see the Calculations and Notifications section in KB00384 for examples.

For additional information and updates to this tech tip, refer to KB00384.