COPYRIGHT (C) 1984-2005 MERRILL CONSULTANTS DALLAS TEXAS USA
Newsletter TWENTY ONE
/* COPYRIGHT (C) 1984-2005 MERRILL CONSULTANTS, DALLAS, TEXAS */
****************NEWSLETTER=TWENTY-ONE***********************************
MXG NEWSLETTER NUMBER TWENTY-ONE March 1, 1992
Technical Newsletter for Users of MXG : Merrill's Expanded Guide to CPE
TABLE OF CONTENTS
page
I. MXG Version Status.
1. Announcement of Production Version MXG 9.9 dated March 1, 1992 2
2. Major enhancements and new products supported in MXG 9.9. 2
3. IBM Announcements and their MXG support. 3
4. What is planned in future MXG Software releases? 3
5. New MXG documentation is planned. 3
II. MVS Technical Notes
1. Type 60 SMF Records filling SMF now fixed by IBM APAR OY43935. 5
2. Type 0 SMF records (IPL) are also created if SMF is RESTARTED! 5
3. MVS/ESA 4.2 TYPE72 CPURCTTM wrong, APAR OY51878 has no PTF yet. 5
4. MVS/ESA 4.2 TYPE72 SYSTRNTM wrong, APAR OY51053 has no PTF yet. 5
5. MVS/ESA 4.2 type 30 records for MASTRJCL with INITTIME wrong. 5
6. ESCON channel utilization probably wrong. 5
7. MVS/ESA 4.2 non-swap block paging may be double counted. 5
8. MVS/ESA 4.2 blocked pages and unblocked pages count questionable. 6
9. TYPE64 VSAM fields wrong values now fixed by IBM APAR OY48286. 6
10. TYPE30 EXCP counts for multi-volume datasets affected by STOPX37. 6
11. A Look at Cache Performance Data for the Amdahl 6100 6
III. CICS Technical Notes
1. CICS user segments and Exclude/Include logic fully supported. 11
IV. DB2 Technical Notes
1. Summary of DB2 2.3 data and measurement changes. 12
2. Using MXG DB2 members ANALDB2R, ANALDBTR and READDB2. 21
V. SAS Technical Notes.
1. MXG STRONGLY RECOMMENDS IMMEDIATE INSTALLATION OF SAS 6.07. 33
2. PROC CONTENTS option DETAILS added in SAS 6.07. 33
3. %MACRO compiler performance improved in SAS 6.07. 33
4. Running out of WORK space produces strange error messages 33
5. Strange errors if MEMSIZE is insufficient. 33
6. SAS 6.06 has been repaired, and can now be installed and used. 33
7. Execution of MXG under SAS Versions 5.18, 6.06, or 6.07 34
VI. MXG Version 9.9 Installation, Space, Compatibility, etc. 37
VII. Documentation of MXG Software. 39
VIII. Change Log - Changes 9.105-9.232 40-72
COPYRIGHT (C) 1992 BY MERRILL CONSULTANTS DALLAS TEXAS
I. MXG Version Status.
1. Production Version is now MXG 9.9, dated March 1, 1992.
MXG Version 9.9 accompanied this Newsletter.
All Changes in this newsletter have been installed in MXG 9.9.
The MXG 9.9 library contains 1,589 members, 383,962 lines of code,
and builds 1030 datasets with 39,435 variables that are documented
in member DOCVER. Datasets changed between MXG 8.8 and MXG 9.9
are documented in member DOCVER09.
2. Major enhancements and new products supported in MXG 9.9.
Support for SAS 6.07 (new options).
Support for Amdahl's SPMS Release 1.2 SMF record.
Support for 4th Dimension Software's CONTROL-D SMF record.
Support for CA-1 (TMS) Release 5 complete rewrite.
Support for CADAM's Statistics Data File.
Support for CICS/ESA 3.2.1 product.
Support for Codework Software's SNAPAD-MVS user SMF record.
Support for Database 2 Release 2.3.
Support for DCOLLECT APAR OY36364 change in track capacity.
Support for Fischer's Totally Automated Office TAO.
Support for HSM 2.6 SMF record.
Support for Interlink's SNS/SNA Gateway SMF record.
Support for Interlink's SNS/TCPaccess SMF record.
Support for Interlink's SNS/TCPvt SMF record.
Support for LEGENT's MIM Multi-Image Manager
Support for LEGENT's LANSPY component of NETSPY (4.1).
Support for LEGENT's BUNDL product modified type 6 SMF record.
Support for MVS/ESA 4.2.2 (RMF 4.2.2) changes.
Support for NetView NPM 1.5 release.
Support for NetView FTP SMF record.
Support for Omegamon for CICS/ESA Version 550 ESRA user SMF.
Support for Omegamon V550 SMF 110 EMPs (Basic, DB2, DL/I).
Support for SCA's CMA-SPOOL user SMF record.
Support for Shared Medical Systems CICS exclude logic.
Support for Softtool Corporations's CCC (Change and Control).
Support for Software AG's NET-PASS user SMF record.
Support for Type 30 APARs OY33312 and OY25606 (no changes).
Support for 3490E (Serpentine) tape device.
Support for 9345E DASD device.
Addition of DB2ACCT,STAT0,STAT1 datasets to your PDB.
Error in VMXGVTOF/VMXGVTOF (only 3 extents kept) corrected.
Revision of BUILDPDB to eliminate SAS 5.18 Compiler Error 344.
Cache RMF Reporter support enhanced, decoded, and documented.
DB2 Reports corrections and enhancements in ANALDB2R.
Fujitsu FACOM MSP and FSP supported in XPDLPDA.
IMS Input Queue time, resources, CONDCODE corrections.
PR/SM Trend Data Base implemented.
VM/XA Trend Data Base implemented.
Each of these enhancements are described in the Change Log, below.
alphabetical list of the most important changes precedes the
changes.
3. IBM Announcements and their MXG support.
IBM has made many major announcements relating to the System/390,
the ES/9000 family, and ESCON capabilities. The following table
lists announced availability dates for the IBM product, and the
corresponding Version of MXG required to support that IBM product.
Product Name Availability MXG Version
Date Required
MVS/370, MVS/XA (all) long ago 8.8
RMF 4.1.2 (for MVS/ESA 3.1.3) Sep 7, 1990. 8.8
RMF 4.2 (for MVS/ESA 4.1) Oct 26, 1990. 8.8
MVS/ESA 4.1 Oct 26, 1990. 8.8
MVS/ESA 4.2 Mar 29, 1991. 9.9
RMF 4.2.1 (for MVS/ESA 4.2) Mar 29, 1991. 9.9
MVS/ESA 4.2.2 Aug 1991. 9.9
RMF 4.2.2 (for MVS/ESA 4.2.2 Aug 1991. 9.9
CICS/ESA 3.1 1990 8.8
CICS/ESA 3.2 Jun 28, 1991. 9.9
DB2 2.2.0 1990 8.8
DB2 2.3.0 Oct 28, 1991. 9.9
VM/ESA 1.1.0 (370 Feature) Oct 26, 1990. 8.8
VM/ESA 1.1.0 (ESA Feature) Mar 29, 1991. 8.8
VM/ESA 1.1.1 Dec 27, 1991. 9.9
4. What is planned in future MXG Software releases?
New release of CICS will be supported upon availability.
Enhancement of AS400 support is planned.
Boole & Babbage CICS/MANAGER will be supported.
Several companies have announced plans for VTAM monitors.
TRIM for ADABAS is in planning; test data did not arrive in time.
Goal Systems EXPLORE/VM support needs corrections.
VM/ESA 1.1.1 has not been tested for all records.
Landmark CICS Version 9 documentation/data was not provided.
Landmark TMVS new version documentation/data was not provided.
JES3 Tape Mount Merge with TYPETMNT is a future consideration.
Cray UNICOS is a distant future consideration.
VAX/VMS Account/SPM is a very distant future consideration.
5. New MXG documentation is planned.
Yes, there will be new printed documentation, which will combine
the 1984 MXG Guide, the 1987 MXG Supplement, and the 700 pages of
MXG Technical Newsletters, plus the notes and text buried in the
various members of the MXG library.
No, the new books will not be completed in 1992. (In fact, we have
just re-printed both the 1984 MXG Guide and 1987 MXG Supplement to
ensure we do not run out of print until the new books are done!)
The complete re-write of MXG documentation began in the fall of 1991
and is still in progress. With over 1,000 MXG-built datasets and
over 40,000 variables, the new Chapter FORTY style documentation
would total over 4,200 printed pages, the equivalent of 6 books the
size of the 1984 Guide, just for dataset documentation! This brief
analysis led to the following plan of attack.
For each "Product" or "Data Source" supported by MXG, there is a
VMAC.... member which reads those data records and creates one or
more MXG datasets. For each of these VMAC.... code members, there
will (eventually!) be an ADOC.... member which documents that data
source. Each ADOC.... member will not only contain the dataset
and variable descriptions now found in sections of Chapter FORTY,
but will also document the "Product" itself. The final format
of each ADOC.... member is still being developed, but each ADOC...
will contain several sections:
Product Information: Vendor, vendor's manuals, how to create the
data records, releases supported, etc.
Dataset Information: Contents of each dataset created by MXG from
this product, like existing Chapter FORTY with
alphabetic list of all variables.
Variable clusters: Grouping of MXG variables by logical grouping
(CPU, I/O, memory, response, owner, etc.). See
page 424 in the MXG Supplement for example.
Report mappings: For important reports from the vendor (eg. IBM
RMF reports), a sample report showing which
MXG variable name creates each report value.
See page 454 in the Supplement for an example.
PRINT/MEANS An edited PROC PRINT of actual observations of
each MXG dataset shows the variable name, its
label, and actual values, which I personally
think is the best way to understand datasets.
A PROC MEANS with minimum and maximum of all
numeric values will be provided also.
Technical reports: Technical papers, discussions, or other useful
information relating to each product will also
be included in that products ADOC.... member.
Currently, 205 "Products" have been identified, and ADOC.... members
will eventually exist for each product. MXG 9.9 contains these new
ADOC members (although none yet contain all of the above sections):
ADOCACHE ADOCDB2 ADOCDB2R ADOCLMS ADOCSPMS ADOCVPD ADOC102
As ADOCs are completed, they will be added to the MXG PreReleases.
In addition to revising Chapter FORTY into the 205 ADOC members, the
other 41 chapters will also be revised, combined, and indexed, and
will be made available online initially and printed subsequently.
At this time, I envision the future MXG printed documentation will
consist of three separate (potentially multi-volume) "books":
a. A "Beginners Guide to MXG Software", subtitled, "What do I do now
that my boss just told me I am the MXG Technican?". This book
will describe how to install, re-install, and use MXG and will
deal exclusively with MXG architecture, execution, etc.
b. A "Documentation of Products Supported by MXG", comprising the
above mentioned ADOC members.
c. A new "Advanced Guide" text comprising the consolidation of the
other forty-one chapters, plus new information.
ALL FUTURE MXG DOCUMENTATION WILL ALWAYS BE PROVIDED INITIALLY IN
MACHINE READABLE FORMAT AS PART OF THE MXG SOFTWARE. Portions of
that online documentation will also be available in printed format.
II. MVS Technical Notes
1. The previously reported problem with Type 60 SMF Records filling
SMF (See Newsletter NINETEEN article on MVS/ESA), has now been
acknowledged by IBM under APAR OY43935, and PTFs now exist to
correct the error. The excess data was created when sites share
systems with DFP 2.4.0 and DFP 3.2.0. DFP V3 added a 'catalog
sharing subcell' to ICFCATALOG's VVR, and a 'large record' was
written that was not needed and not used for catalog recovery,
and the PTF now no longer writes the record when only the catalog's
VVR sharing subcell was updated.
2. Type 0 SMF records (IPL) are also created if SMF is RESTARTED!
The only way to tell that this type 0 is a "bogus" IPL event is to
look for a type 90 SMF record with SUBTYPE=14 (SETSMF RESTART SMF)
event code that was written at the same time as the type 0 record.
It's not clear if this is a bug or a feature, but the unexpected
IPL record when no IPL occurred confused Dan Kaberon! Alternatively
you could use the TYPE90 with SUBTYPE=9 and SUBSYS='SYS' to identify
only actual IPLs (and exclude RESTART SMF events).
3. MVS/ESA 4.2 CPURCTTM (Region Control Task CPU time) in TYPE72 is
enormously wrong (TYPE30 data is fine). While some RMF intervals
have reasonable data, there are intervals with impossible values
(30 minutes in RCT alone in a 30 minute interval!), and comparison
of RCT in the type 72 shows significantly more time than in the type
30 record. This problem has been seen by sites with RMF and CMF;
the error is in SRM calculation of WAMPRCT. This problem was opened
at the IBM support center in July, 1991, but APAR OY51878 was not
issued until Feb 6, 1992, and no PTF yet exists for that APAR.
MXG 9.9 circumvents the error's impact on TYPE72 variable CPUTM by
subtracting CPURCTTM from CPUTM in MXG exit EXTY72, but when you
have installed the PTF for this APAR, you will need to remove the
circumvention from member EXTY72.
4. MVS/ESA 4.2 TYPE72 field SYSTRNTM (SMF72TST) is also erratically
wrong, but as this is a new measure of "transaction duration",
this is not quite as critical as CPU time. This problem was at the
IBM Support Center since July 1991, and was finally accepted as
APAR OY51053 in January, 1992 (although no PTF yet exists!).
5. MVS/ESA 4.2 has type 30 records for MASTRJCL with Initiate Timestamp
greater than ALOCTIME and LOADTIME. APAR OY49418 accepts the error,
and PTF UY74921 now exists.
6. Boris Ginis, BGS, has noted that with ESCON channels, the channel
path utilization is about 10% greater than the utilization that is
inferred by summing the connect time of the LCUs using that channel.
Prior to ESCON channels, the delta was only about 1%. In both cases
the channel utilization was on the order of 50% busy. Another site
noticed similar values, and were informed that IBM thinks there is
probably an error in the sampled percent channel busy measurement
with ESCON, but that the ESCON connect time is accurate. This does
does seem reasonable to me; can anyone else confirm these as fact?
7. Boris also suspects that non-swap block paging may be double counted
in RMF 4.2.1. The type 71 record's sum of page ins and page outs
for swap, non swap, blocked and unblocked pages shows about 16-17
pages per second higher than the paging rate calculated from the
type 75 total pages transferred, and the non-swap block paging rate
in the same interval is just about 16-17 pages per second. Non-swap
block paging is BLKSAUIN+PGBKAUIN (SMF71BLK+SMF71BLP). See below.
8. Diane Epstein noted that the blocked pages and unblocked pages count
in TYPE71 did not seem to match the values reported by IBM on their
RMF Paging Activity report. It turns out that PVTNPIN (SMF71PIN)
includes the number of blocks-paged-in, BLKSAUIN (SMF71BLK), because
when the first page in a blocked-page-in-request is requested, the
system does not know it is part of a block of pages. This means
that the number of non-blocked non-swap page-ins must be calculated
as PVTNPIN-PVTCAIN-BLKSAUIN,and the number of blocked non-swap
page-ins is calculated as PGBKAUIN+BLKSAUIN, and pages per block
is calculated as (PGBKAUIN+BLKSAUIN)/BLKSAUIN.
9. Gross errors in TYPE64 VSAM fields have been reported by several
sites that appear to be corrected with APAR OY48286. Values in the
actual SMF type 64 record were FFFFFF06, indicating a negative value
in IBM's terms, but reading that EXCP count with SAS as PIB4. gives
a value on the order of 4,294,000,000! The errors were not only in
EXCP fields, but also DELETES, INSERTS, UPDATES, CISPLITS, CASPLITS,
and RETRVALS.
10. EXCP counts in type 30 records for multi-volume datasets with the
STOPX37 product are not correctly assigned to the correct DDNAME.
The EXCP counts are captured in the DD segment, but when STOPX37
goes to multiple volumes, it dynamically allocates the device with
system-assigned DD names (SYS00001, SYS00002, etc., for each volume)
and the EXCP count is put in the segment for the dynamic DD name
instead of the real DD name for the multi-volume dataset. STOPX37
is aware of this condition, and are investigating.
11. A Look at Cache Performance Data for the Amdahl 6100
Dick Sziede, PRC Inc., 703 883-8551
Abstract
Instrumentation is a key component of any performance oriented product.
Cache memory in the Amdahl 6100 DASD controller is instrumented by its
controlling software, Storage Products Management Services (SPMS). The
performance data recorded by SPMS Release 1.2.11 is much improved over
data recorded by prior versions. These data do not contain sufficient
information to permit performance or capacity management.
Background
Storage Products Management Services is the external interface between
Amdahl storage controllers and the systems programmer. SPMS reports
performance data in two ways: a spun-off SYSOUT report, and through SMF
records. Samples of SYSOUT and SMF data are contained in ADOCSPMS.
Poor documentation of the data recorded by SPMS R.1.0.92 caused the
author considerable concern. Many of these concerns have been
addressed in R.1.2.11. The first section of this paper covers fixed
problems. It is included to allay fears of anyone who might have seen
the (unpublished and somewhat acerbic) version of this paper that dealt
with R.1.0.92 of SPMS.
The second section covers problems that are new, or have not been
fixed.
The last section reiterates the author's principal concern: it remains
impossible to develop a complete picture of what is going on in the
6100 cache. It is not possible from the data recorded to tell if the
cache is over committed, or thrashing.
Section 1. The Good News
Whole bunches of stuff have been fixed.
Documentation of the prior release of SPMS' SMF record consisted of an
80-80 list of the DSECT that mapped it. Period.
With R.1.2.11, the DSECT is provided machine-readable, as is a SAS
routine to decode it. Clear explanations of most of the fields appear
in the -006 version of the SPMS User Guide. The new records are
supported by Release 9.3 of MXG. At this writing, there is no
supported MICS component. The PRINTS in ADOCSPMS are from MXG R.9.3.
The whole dataset name of datasets cached by name now appears in
reports and in the SMF record. Thank goodness! An occasional dataset
name is truncated when the continuation line is lost, but who quibbles?
See ADOCSPMS for the SYSOUT report.
The extent ranges bug is now fixed. SMF and the SYSOUT reports now
agree on which extents are being cached.
Controller ID is now recorded. However, the size of the cache and of
the NVS in the controller is not recorded. Amdahl says to do this
would require an engineering change. This is hard to understand,
because these very data appear in start-up and status messages:
SPMS840I SPMS Version 1.2.1.1 Task Status Information
SPMS847I CTL 30 A(Yes),VOLS(10),EXTS(14),NVS(Installed
and Enabled),CACHE(Ready)
SPMS835I Cache(64 MB,0 Deg Blks),NVS(8 MB,0 Deg Blks)
SPMS847I CTL 40 A(Yes),VOLS(10),EXTS(16),NVS(Installed
and Enabled),CACHE(Ready)
SPMS835I Cache(64 MB,0 Deg Blks),NVS(8 MB,0 Deg Blks)
The sequential-detect fields are now documented, and the workings of
the sequential-detect algorithm are in the user guide. 6100 R3
firmware uses access history to detect sequential I/O. The alternative
would have been from the Define Extent CCW, but at this time, the 6100
does not implement ECKD commands such as Define Extent. (The recently
announced Amdahl 6390 disks will require ECKD).
Interval Start Time and Interval End Time are now in the SMF record.
This means the first record cut after an SPMS start-up, or after the
operator diddles with the LOGGER, is no longer useless.
SPMS Release ID is now in the SMF record.
The full dataset name, the serial number, the device number, and
controller ID are now in the SMF record. Now SPMS records can be
correlated with RMF, at least over long periods. SPMS still doesn't
sync with the RMF interval.
We now know what is meant by SUBSYSTEM-ID. This is the actual
subsystem name for SPMS, coded in IEFSSN. Alas, the string SPMS is
hard-coded. This does not allow for multiple versions, or conformity
with a local naming convention.
SPMS now remembers the parameters specified when the LOGGER was cold-
started. It no longer falls back to defaults if one closes and
restarts the LOGGER. In fact, warm-start rather than cold-start is now
the recommended way of starting SPMS.
Section 2. The Not So Good News
The following problems have been reported to Amdahl, and are being
tracked under Service Order 513270.
What is ALLOCATED-TRACK-BLOCK-COUNT? Why do I sometimes get a negative
number in this field? This bug is still with us from previous
releases. My Amdahl rep thought that the sum of all ATBCs over an
interval would make sense: that ATBC is a running tally that'd
aggregate to the sum of the "working sets" for the extents. This
didn't work with my numbers. ATBC varies by orders of magnitude from
the highest possible number for my controllers.
I know from the @STATUS command that my two 6100 controllers are named
30 and 40. How did they get these names? Can I change them to Spot
and Rover? Amdahl: No. The controller names are numeric. The FE can
change them, but with difficulty.
There is a new bug. SPMS appears to loose track of time. The interval
that spans midnight ends before it began. Downstream code sees this as
an interval with negative duration.
There are two reasons for SPMS to cut an SMF record: MONITOR activity,
and LOGGER intervals. The LOGGER creates SMF records at specified
intervals. The MONITOR makes records at event boundaries.
The relationship between records cut at LOGGER intervals and at MONITOR
intervals is unclear. MONITOR and LOGGER not only do not sync to RMF,
but don't sync to each other. LOGGER records all have the same SMF
timestamp. One hopes this means that the records constitute a
"snapshot" of the controller stats at that point in time.
It would be ideal were the LOGGER to slave its interval to that of RMF.
Amdahl points out that the automatic commands in the next release can
be used to sync recording to a standard time and interval. This almost
works. It would not be much more difficult to go the rest of the way.
RMF interval parameters are available in virtual memory, or if one is
shy about prowling through control blocks, in SYS1.PARMLIB. SPMS is a
subsystem: it gets a look-see at all console traffic. It can slave
itself to RMF even if the operator changes the RMF interval. Look for:
F RMF,F ZZ,INTERVAL(...
MONITOR records are supposed to be cut when something changes: a
dataset is added, deleted, or goes into extents. They also appear at
intervals. What is the relationship of the counts in MONITOR and
LOGGER? Are they independent statistics? Or, are the MONITOR counts
INTERVAL-to-date? If the latter, what happens if I set the MONITOR
interval higher than the LOGGER?
Member ADOCSPMS prints the MONITOR and LOGGER data for one volume. The
timestamps and counts for these records suggest that the LOGGER and
MONITOR data are not independent. Rather, it looks like either MONITOR
or LOGGER, when they cut a record, reset the 6100 counters to zero. If
this is so, why do we have both?
MONITOR should cut records for Extent Changed, or for Extent Deleted:
reason codes 1 & 2. But, only when these events occur. Producing a
MONITOR Stopped record, reason code=3, at every MONITOR interval is a
bug, that will be fixed in R.1.2.12.
The SMF record contains segments for Cache Extent Definition (CED) and
for each real extent (EXT). It is necessary to collect statistics for
both extents and extent definitions, because the relationship between
these entities need not be isomorphic. For example, an extent
definition could be a dataset. That dataset could have multiple
physical extents, even spanning volumes. Similarly, the controller
will merge extents from contiguous extent definitions, and manage the
merged extent as a unit.
The statistics in the Cache Extent Definition part of the SPMS record
are always zero. MXG deals with this by summing the extent stats into
the extent definition stats, but I'm not sure this is what was intended
by Amdahl. Amdahl considers this a bug, to be fixed in R.1.2.12.
The SAS sample code provided with SPMS has an implicit assumption that
there will be only one EXT segment per CED segment. This is not the
way the doc reads. This is not what the supplied mapping macro
implies. (MXG is written to allow tuples of the EXT segment. One
believes that this is what was intended).
Section 3. The Bad News
Some data just aren't there.
Rich Olcott has said that each layer in the storage hierarchy is a
cache for the layer below it. One can draw a strong analogy between
the disk/cache relationship, and that of real memory to virtual memory.
With this analogy, one can imagine the sorts of data one would need to
manage performance and capacity for a caching controller.
Consider the RMF data collected on behalf of the ASM and RSM in MVS.
What is needed is stats similar to those collected for real and virtual
memory management in MVS. The paging statistics, workload MSO, and
memory allocation statistics collected by RMF provide the example. In
particular, we need the parameters of the 6100's LRU algorithm just as
we know the parameters of working-set and page- stealing in MVS' memory
management.
There has to be some analog to UIC, and Available Frame Count that will
tell how much the cache resource is stressed. There should be a
measure analogous to working-set, taken by cache extent definition,
that tells how much cache is absorbed by each dataset put behind it.
We need data to answer the questions, "Am I thrashing my cache?" and
"How close am I to thrashing my cache?"
The statistics that may correspond to UIC and AFC in RMF are average,
high, and low cache occupancy. There should be a working-set measure.
Cache occupancy is the integral of working-set over time. A field
called Allocated-Track-Block-Count may well be the working-set analog,
but since it shows up negative as recorded by my 6100's, it is a trifle
hard to interpret. Alas, cache occupancy is not being recorded at all.
Amdahl has suggested that the flawed ATBC will be removed in future
releases. This is a step backward. Let this paper serve as a request
to Amdahl that ATBC be corrected, and some form of the other needed
statistics be recorded.
The number of pinned tracks isn't recorded. As my datacenter moves to
a 24/7 service schedule, I may have to run with pinned tracks for quite
a while before I can get a service window. Pinned tracks may well be a
performance datum, rather than an event to be fixed immediately. It
needs to be recorded.
Amdahl offers an event simulation model for cache tuning. CSIM is
calibrated with GTF CCW trace data. It will generate a detailed
picture of cache activity given a variety of configurations. This is a
valuable and important tool, but not a substitute for the measurements
this paper requests. An analogy may illustrate why.
It is possible, with some engineering knowledge, a stopwatch, and a
measured mile, to make measurements to calibrate a computer model of
automobile performance. With such a model, one could predict the
velocity of an automobile from how far the gas pedal is pushed down.
With careful selection of input measures, such a model could be made
arbitrarily accurate. But would you choose such a model over a
speedometer?
Conclusions
The 6100 is a superior product. SPMS, although it has come a long way
since the last release, isn't. The 6100 is an expensive product. SPMS
should be the management tool that enables users to get their money's
worth out of the 6100 cache. The performance data recorded by SPMS are
less than adequate. Capacity planning data are almost non- existent.
Capacity planning data look like this:
|
C | Cache Installed
A | +----------------------
C | | o
H | | o
E | | o o
| |
D | -----------+ o o o
E | o
M | o o
A | o
N | o
D | o
| o o
|
|
+----------------------------------------------------
WORKLOAD
What is missing, is something to plot on the Y-axis.
The points in the graph must be measurements, not estimates. In the
scenario implied by the figure, the capacity manager uses the 6100
capacity data asked for in this paper to forecast cache demand as a
function of workload. He then schedules a cache upgrade before
capacity is exceeded. This is what is meant by the jog in the "Cache
Installed" line: the customer buys more stuff from Amdahl.
As SPMS is now, this scenario is impossible. The data graphed on the
Y-axis don't exist. In a real-life scenario, the capacity manager
cannot know cache capacity is over-committed until it happens. He
will find out from declining hit ratios, and degraded service times.
When this happens, it is too late. Under these circumstances, without
measurements to show the true problem, management may be inclined to
replace the 6100 with other technology, rather than upgrade a failing
device.
Performance of the Automated Patent System hardware, depends heavily on
getting maximum benefit from the cache on the 6100 controllers. We
have no choice but to do exactly that.
III. CICS Technical Notes
1. CICS user segments and Exclude/Include logic fully supported
User segment for vendor products added/enhanced
User clocks/counters/characters segments can be added to CICS type
110 transaction records; now their order of processing can be set
by new member IMACICDA, based on the actual order in your CICS MCT.
IMACICDA also can be used to tell MXG which APPLIDs (CICS regions)
have which data segments. Note that while the order of segment
processing is now set by IMACICDA, it is still necessary for you to
actually enable segment processing by removing a comment block in
each of the user segment members you wish to enable. MXG 9.9 knows
about the following user segments:
IMACICDL IBM Local DL/I counters.
IMACICDU Other user data (length still set in IMACICUS).
IMACICDB IBM DBCTL counters, CICS/ESA only.
IMACICOC Omegamon OMEGBSC (Basic) segment, CICS/ESA only.
IMACICOL Omegamon OMEGDLI (DL/I) segment, CICS/ESA only.
IMACICOB Omegamon OMEGDB2 (DB2) segment, CICS/ESA only.
How do you know what user segments exist? Run MXG's UTILCICS and
look at the columns CMODNAME and CMODHEAD. CMODNAME is the module
that creates the data, and CMODHEAD is the "variable" name created.
If you look at the end of the transaction record (usually the last
IBM field is the 72nd, with CMODNAME=DFHDEST and CMODHEAD=TDIOWTT).
Following that field you may see these optional segments:
MXG member CMODNAME CMONHEAD length
IMACICDL ???????? ???????? 68
IMACICDU USER-CHOICE USER-CHOICE ??
IMACICDB DBCTL RMIDATA 256
IMACICOC OMEGBSC OMEGAMON EMP 116
IMACICOL OMEGDLI OMEGAMON DLI EMP 92
IMACICOB OMEGDB2 OMEGAMON DB2 EMP 100
(The base CICS 3.1 record is 380 bytes, 3.2.1 is 388).
CICS Include/Exclude logic is now fully supported.
New member IMACEXCL supports CICS transaction records that have been
modified by the use of Exclude/Include logic in the CICS MCT. There
are two specific vendor products are explicitly supported, and you
can also define your own modifications in this record. These macros
are defined in IMACEXCL for you to change to meet your needs:
_CICXLTR - "Enabling" macro selection which controls whether the
exclude processing applies for all regions, or only
for certain specified CICS APPLIDs.
_CICXCTR - "Code" macro selection which specifies which of the
following "Exclude code" macros will be used:
_CICXCOM - for OMEGAMON for CICS Version 2
_CICXCSM - for Shared Medical Systems
_CICXCUS - for "user" defined excludes.
Instructions for both of these enhancements are contained in comments
in each of the above IMAC.... members.
IV. DB2 Technical Notes
1. Summary of DB2 2.3 data and measurement changes.
Type 101 accounting records are compatible and previous versions of
MXG will not fail with DB2 2.3 records, but processing SMF type 100
or 102 records will cause prior versions of MXG to fail, because
some fields were expanded in place and other new fields inserted.
However, the new, important, and needed information that IBM added
more than makes up for the minor inconvenience of a new MXG version.
Contents
a. Matching DB2 transaction to CICS transactions is now possible.
b. The new concept of a "Package".
c. IFCIDs 147 and 148 triplets multiple header pointers.
d. What's not completed in MXG Version 9.9.
e. Detail SMF Record structure and control block names.
f. Header Changes that apply to all records.
g. DB2STAT0 - Type 100 Subtype 0 SMF Record changes.
h. DB2STAT1 - Type 100 Subtype 1 SMF Record changes.
i. DB2ACCT - Type 101 Subtype 0 SMF Record changes.
j. T102Snnn - Type 102 IFCIDs SMF Record changes.
a. Matching DB2 transaction to CICS transactions is now possible.
DB2 2.3 allows matching DB2 transactions with their CICS counterpart,
because the DB2 record now contains the CICS Logical Unit of Work ID
information. However, there are several "LUWID" fields in DB2, and
none exactly match field-for-field their CICS counterparts. Read on:
In the standard header, QWHS, the QWHSLWID is a 24 byte field that
identifies the source of the DB2 transaction. However, this LUWID
does NOT change when thread reuse is used, and thus the QWHS LUWID
CANNOT be used to match to each CICS transaction record.
24 QWHSLWID Logical Unit of Work ID
|---------------------------------------------------------|
8 8 6 2
|------------|------------|-----------------|-------------|
Network ID LU Name Instance Number Commit Count
QWHSNID QWHSLUNM QWHSLUUV QWHSLUUC
To solve the thread reuse problem, you must specify the TOKENE=YES
parameter in your CICS RCT so that a DB2 accounting record will be
created for every CICS transaction, and the unique LUWID for each
CICS transaction will be in your DB2 data. The QWHS LUWID fields
will not change with thread reuse, but the new QWHCTOKN field, added
to the correlation header, contains the DB2 LUWID that DOES change
for each CICS transaction, even when thread reuse occurs. However,
the QWHCTOKN is only 22-bytes long; 16-bytes for the Network name,
and 6 bytes for the unique instance number.
22 QWHCTOKN
|-------------------------------------------|
Prior to DB2 2.3, the distributed header formerly contained the DB2
LUWID, but these fields no longer exist (and are listed here just for
completeness in case you had previously used them), having been now
replaced by the QWHS and QWHC fields.
24 QWHDLUWI Logical Unit of Work ID
|---------------------------------------------------------|
8 8 6 2
|------------|------------|-----------------|-------------|
Network ID LU Name Instance Number Commit Count
QWHDNETI QWHDLUNM QWHDUNIQ QWHDCCNT
The QWAC DSECT also previously held parts of the DB2 LUWID, which are
now redundant with the QWHS/QWHC fields, but for completeness:
16 4
|-------------------------| |-------------|
Network ID + LU Name Commit Count
QWACNID QWACCOMM
If re-signon has occurred with an identical authorization-id, the
value of QWACRINV (the field which indicates why accounting was
invoked) will be set to 6.
So the bottom line of the DB2 side of the match is to use QWHCTOKN.
But now, let's look at the CICS equivalents, which have preceded
their addition to DB2 by several years. The CICS NETSNAME and UOWID
fields are used to match all parts of CICS MRO transaction, and these
same two CICS fields are logical contained in the new QWHCTOKN, but
it is not straightforward! First, looking at the CICS fields, the
CICS NETSNAME is a 20-byte field, and
NETSNAME contains: if the originating terminal is:
netname local terminal, from TCT
networkid.Luname VTAM ISC LUTYPE6.2 or IRC link
networkid.generic_applid non-VTAM or ISC LUTYPE6.1
jobname.stepname.procname DL/I batch session
and NETSNAME is padded by three bytes which may or may not be nulls.
The periods (EBCDIC hex 4B) shown above are actually in the first 17
bytes of NETSNAME! Unfortunately, the new DB2 QWHCTOKN variable has
only 16 bytes for the NETSNAME part of the match, and those 16 bytes
do not contain periods. QWHCTOKN does contain the network name in
the first 8 bytes (padded with blanks), and the LU name is contained
in the second 8 bytes (also padded with blanks). (IBM has not stated
what QWHCTOKN will contain for DL/I batch source.)
The remaining 6 bytes of QWHCTOKN are the DB2 "Instance Number"
or "Uniqueness Value", which is actually a timestamp value, although
IBM states "though this field may appear to be a timestamp, it is
not to be processed as one." This timestamp is passed into DB2
from CICS, and in MXG CICS data this has been stored as the UOWTIME,
the first 6 bytes of CICS's 8-byte UOWID (Unit of Work ID). The last
two bytes of CICS's UOWID is the count of sync points, just like the
final two bytes of DB2's LUWID is the number of commits, and cannot
be used for matching, because it is not constant across transactions.
MXG thus creates two variables, NETSNAME and UOWTIME from the new DB2
QWHCTOKN in the DB2ACCT dataset so that DB2 transactions in DB2ACCT
can be exactly matched to CICS transactions in CICSTRAN.
b. The new concept of a "Package", which is a lower granularity than
a PLAN, is captured in a number of type 102 IFCID's records:
In IFCIDs 29,30,31,53,58-66,124,145, and 148, a new 60-byte area
overlays a collection of some-old, some-new, and some-expanded
fields. This 60-byte "Current Package Name" consists of:
0CL60 "Current Package Name" CP
|--------------------------------------------------------------|
16 0CL44 "Package Name" PK
|----------|---------------------------------------------------|
Location
Name 18 18 8
|-----------------|--------------|------------------|
Collection Name Package ID Consistency Token
or or or
Collection ID Program Name Timestamp
(Program Name was 8-bytes)
but in IFCIDs 108, 110, and 177 IFCIDs, a 126-byte "superset" field
is defined that uses the same "Package Name" and PK suffix as the
above 44-byte part of the 60-byte "Current Package Name"! MXG has
labelled the 126-byte variables "Current Package and Version Name".
0CL126 "Current Package and Version Name" PK
|--------------------------------------------------------------|
16 18 18 8 0CL66 VR
|--------|----------|-------|-----------|----------------------|
Location Collection Package Consistency 2 VL 64 VN
Name ID ID Token |-------|--------------|
Version Version
Length Name
Unfortunately, the IBM field names of the sub-components are
somewhat inconsistent, both vertically and horizontally:
IFCID: 29-31 53 58-66 64 108 110 124 145 148 177
16-byte Location Name LN LN LN LN NL PL LN LN LN LO
18-byte Collection Name CI PC PC CI NC PC CI PC CI CO
18-byte Package ID PI PN PN PN NI PI PI PN PN PI
8-byte Consistency Token CT TS TS TS NT PT CT TS CN CT
2-byte Version Length VL VL VL
64-byte Version Name VN VN VN
c. IFCIDs 147 and 148 triplets R5O and R7O point to headers QWHC and
QWHD that appeared to be redundant to those in the product section,
but these two sets in these IFI data relate to the agent doing the
monitoring in the product section, while the agent described in the
R5O and R7O sections is the agent actually being monitored (e.g.,
the product agent is the online monitor, the R5O/R7O pair describe
your TSO session that is using DB2). MXG keeps the identification
of the latter agent, the one being monitored, for these IFCIDs.
d. The following new IFCIDs are not yet decoded by MXG, because no
sample data records have been created (perhaps too obscure?):
126 IFI 127 TC4 128 TC4 129 IFI (130-139 do not exist)
172 ST3 177 TC3 180 GC9 181 GC9
182 GC9 (184,185 do not exist)
186 UC (187,188,189 d.n.e.)
190 GC5 192 ST4 193 ST4 194 ST4
195 ST4
e. DB2 2.3 SMF Record Changes - Details and more details!
DB2 SMF/GTF Records are documented in a multitude of DSECTS in the
DB2 Macro Library that comes with the IBM Product. You should ask
your DB2 person the name of the library, and view the members online
if you need additional information. The important members are:
Member DSNWMSGS - documents the individual fields in the DSECTS by
field name (which of course is the MXG Variable name). This member
is quite extensive in its detail, often containing performance
discussions for large or small values of particularly critical
fields. Simply search for the MXG Variable name of interest. (Note
that MXG expands the QBS... buffer fields into four variables QB1...
QB2... QB3... and QB4... for each of the four buffers). Eventually
these descriptions will be in MXG's "Chapter 40" section for DB2
datasets, when completed.
The three records (100, 101, 102) are combinations of individual
DSECTS in the IBM Macro Library that start with DSND.... and end with
suffixes:
Record Header (all): SMF = QWSP GTF = QWGT
Record Type-Subtype: 100-0 100-1 101 102
IFCIDs: 0001 0002 0003 0004-0195
Structure Defined: QWST QWST QWAS
Self Defining: QWS0 QWS1 QWA0 QWT0
DB2 Headers: Standard QWHS QWHS QWHS QWHS
Correlation QWHC QWHC QWHC QWHC
Trace QWHT QWHT QWHT QWHT
CPU QWHU QWHU QWHU QWHU
Distributed QWHD QWHD QWHD QWHD
Data sections: QWSA QXST QWAC QW00
QWSB QTST QXST QW01
QWSC QBST QBAC QWPZ
Q3ST QIST QTXA QW02
Q9ST QTXA QLAC
QWSD QISE
QVLS
QVAS
QSST
QLST
QJST
QDST
f. Type 100/101/102 Header Changes:
QWS0 - Offsets
Reserved triplets QWHSRSV3 now OFFQDST (RCO/RCL/RCN) for
Type 100 Subtype 1.
Note: Error in IBM Documentation. DSNDMSGS line 364-366 has
RCO/RCL reserved instead of DSNDQDST just for RCN.
QWHS - Standard Header
LU 6.2 variables, long needed to match DB2 activity directly to
the CICS/IMS transaction, were formerly only in the Distributed
Header, but have now been moved to the Standard Header:
Description Standard CICS Distributed
Header Name Name Header Name
NETWORK*ID QWHSNID NETNAME (part) QWHDNETI
LUNAME QWHSLUNM NETNAME (part) QWHDLUNM
INSTANCE*NUMBER QWHSLUUV UOWID (part) QWHDUNIQ
COMMIT*COUNT QWHSLUCC UOWID (part) QWHDCCNT
The Logical Unit Of Work ID (called LUWID in DB2, called UOWID in
CICS) defined for the LU 6.2 interface can now be used to match
a DB2 instance to its CICS or IMS counterpart. The DB2 LUWID
uniquely identifies the thread within the network and consists of
a. The fully qualified network identification, consisting of
QWHSNID and QWHSLUNM (two 8-byte fields).
b. An Instance Number QWHSLUUV,a 6-byte hex datetimestamp.
(IBM goes out of their way to tell you not to treat this as a
timestamp, in my opinion, because they want to be free to
change its value, and they don't want to have define/support
the event when the timestamp is taken.)
c. an LUW Sequence Number QWHSLUCC that is used to uniquely
identify the last COMMIT scope in which the logical unit
participated in. IBM notes that the -DISPLAY THREAD command
does not display the COMMIT count, and QWHSLUCC only reflects
an accurate count when DB2 is acting as a server of another
DB2, and for a remote unit of work QWHSLUCC is set to one
before any commits have occurred and thus does not record
commit activity.
QWHC - Correlation Header
One long-needed variable, QWHCATYP, was added to uniquely define
where this DB2 transaction came from:
QWHCATYP='CONNECTING*SYSTEM TYPE*CODE'
1='1:TSO'
2='2:DB2 CALL ATTACH'
3='3:DL/I BATCH'
4='4:CICS ATTACH'
5='5:IMS ATTACH BMP'
6='6:IMS ATTACH MPP'
7='7:DISTRIBUTED UOW'
8='8:REMOTE UOW'
9='9:IMS CONTROL REGION'
One additional, quite significant new field was also added:
QWHCTOKN='ACCOUNTING*TOKEN'
QWHD - Distributed Header
New variables (3) added to Distributed Header:
QWHDPRID='APPLICATION*REQUESTOR*PRODUCT ID'
QWHDSVNM='SRVNAM*PARAMETER*OF DRDA EXCSAT'
QWHDTSTP='DBAT TRACE*RECORD*TIMESTAMP'
Old variables (4) that were in Distributed Header were removed
(and renamed into the Standard Header as described previously.
g. DB2STAT0 - Type 100 Subtype 0 SMF Record changes:
QWSA - No change.
The order of the (up to) four QWSA sections, one for each of the
"Resource Manager and Control Address Spaces" CPU times are:
1st- Master
2nd- DBM1
3rd- IRLM if there are only 3 segments.
If there are 4 segments, the 3rd is DDF and the 4th is IRLM,
and there are 4 segments only in a DDF environment.
QWSB - No change.
QWSC - No change.
Q3ST - No change.
Q9ST - New variable (compatibly added):
Q9STCTRM='ARCHIVE*LOG*COMMANDS'
QWSD - No Change.
QVLS - No Change.
QVAS - No Change.
QSST - No Change.
QLST - New variables (5):
QLSTCBLB='SWITCHES FROM*CONT BLOCK TO*LIMITED BLOCK MODE'
QLSTRBND='SQL STATEMENTS*BOUND FOR*REMOTE ACCESS'
QLSTBROW='MESSAGE*BUFFER ROWS*IF BLOCK FETCH USED'
QLSTBTBF='BLOCKS*TRANSMITTED*USING BLOCK FETCH'
QLSTBRBF='BLOCKS RECEIVED USING BLOCK FETCH'
QJST - No Change.
QDST - New Control Block.
Only one new variable:
QDSTQDBT='DBAT QUEUED*DUE TO MAX*RMT CONCUR THREADS'
h. DB2STAT1 - Type 100 Subtype 1 SMF Record changes:
QXST - New Variables (4):
QXSETHV ='SET*HOST-VARIABLE*STATEMENTS'
QXALDAB ='ALTER*DATABASE*STATEMENTS'
QXDRPPKG='DROP*PACKAGE*STATEMENTS'
QXDSCRTB='DESCRIBE*TABLE*STATEMENTS'
QTST - New Variables (25):
QTAUPUB ='SUCCESSFUL*AUTH CHECKS*EXEC AUTHORITY'
QTAUTOBA ='ATTEMPTS*TO AUTOBIND*A PACKAGE'
QTBINDPA ='BIND (ADD)*PACKAGE*SUB-COMMANDS'
QTBINDPR ='BIND (REP)*PACKAGE*SUB-COMMANDS'
QTCURPB ='PB COUNT*CURRENTLY ON*FREE PB CHAIN'
QTDSDRN ='DDS THAT*DRAIN HAS*PRODUCED '
QTEXDRN ='TIMES THAT*DRAIN PROCESSING*COMPLETED'
QTFREEAP ='ATTEMPTS*TO FREE*A PACKAGE'
QTFREEP ='FREE*PACKAGE*SUB-COMMANDS'
QTMAXPB ='MAXIMUM PB*COUNT ON*FREE PB CHAIN'
QTOPNNO ='OPEN FAILURE*USING*DRAIN PROCESS'
QTOPNOK ='OPEN SUCCESSES*USING*DRAIN PROCESS'
QTPKABND ='PACKAGES*AUTOBOUND'
QTPKALL ='PACKAGES*ALLOCATED'
QTPKALLA ='ATTEMPTS*TO ALLOCATE*A PACKAGE'
QTPKGBD ='PACKAGES*BOUND'
QTPKGFRD ='PACKAGES*FREED'
QTPKGRBD ='PACKAGES*REBOUND'
QTPUBDD ='DDS USED*FOR CLOSE(NO)*TS'
QTRBINDP ='REBIND*PACKAGE*SUB-COMMANDS'
QTRBNDPA ='ATTEMPTS*TO REBIND*A PACKAGE'
QTREOPN ='REQUESTED PB*FOUND ON FREE Q*DURING OPEN'
QTSLWDD ='DDS USED*FOR SLOW CLOSE*TS'
QTSTDRN ='DRAIN DISABLED*NO TRIGGER*REQUIREMENT MET'
QTTTDRN ='TIMES THAT*DRAIN HAS BEEN*TRIGGERED'
QBST - INCOMPATIBLE CHANGES:
Previous variables deleted and overlaid:
(IBM Field Names QBSTWMAX QBSTPFDC for each buffer pool)
QB1TPFDC='1ST TIMES*PREFETCH DISABLED*CONCURRENT'
QB1TWMAX='1ST WK FILE*NOT CREATE*INSUF BUFFERS'
QB2TPFDC='2ND TIMES*PREFETCH DISABLED*CONCURRENT'
QB2TWMAX='2ND WK FILE*NOT CREATE*INSUF BUFFERS'
QB3TPFDC='3RD TIMES*PREFETCH DISABLED*CONCURRENT'
QB3TWMAX='3RD WK FILE*NOT CREATE*INSUF BUFFERS'
QB4TPFDC='4TH TIMES*PREFETCH DISABLED*CONCURRENT'
QB4TWMAX='4TH WK FILE*NOT CREATE*INSUF BUFFERS'
New Variables (7 for each buffer pool):
QB1TLPF ='1ST LIST*PREFETCHES*REQUESTED'
QB1TMAX ='1ST BUFFERPOOL*NOT SUPPORT*CONCURR WORKFILES'
QB1TWBVQ='1ST PAGES*DEQUEUED FOR*DESTRUCT READ'
QB1TWDRP='1ST PAGES FOR*DESTRUCTIVE*READ REQUESTED'
QB1TWFD ='1ST WORKFILES*DENIED*DURING SORT/MERGE'
QB1TWFF ='1ST SORT/MERGE*INEFFICIENT*(BUFFER SHORTAGE)'
QB1TWFM ='1ST MAX WORKFILES*ALLOCATED*DURING SORT/MERGE'
QB1TWFR ='1ST REQUESTS TO*QUERY FOR WORKFILES*IN SORT/MERGE'
QB1TWFT ='1ST WORKFILES*REQUESTED*DURING SORT/MERGE'
QB2TLPF ='2ND LIST*PREFETCHES*REQUESTED'
QB2TMAX ='2ND BUFFERPOOL*NOT SUPPORT*CONCURR WORKFILES'
QB2TWBVQ='2ND PAGES*DEQUEUED FOR*DESTRUCT READ'
QB2TWDRP='2ND PAGES FOR*DESTRUCTIVE*READ REQUESTED'
QB2TWFD ='2ND WORKFILES*DENIED*DURING SORT/MERGE'
QB2TWFF ='2ND SORT/MERGE*INEFFICIENT*(BUFFER SHORTAGE)'
QB2TWFM ='2ND MAX WORKFILES*ALLOCATED*DURING SORT/MERGE'
QB2TWFR ='2ND REQUESTS TO*QUERY FOR WORKFILES*IN SORT/MERGE'
QB2TWFT ='2ND WORKFILES*REQUESTED*DURING SORT/MERGE'
QB3TLPF ='3RD LIST*PREFETCHES*REQUESTED'
QB3TMAX ='3RD BUFFERPOOL*NOT SUPPORT*CONCURR WORKFILES'
QB3TWBVQ='3RD PAGES*DEQUEUED FOR*DESTRUCT READ'
QB3TWDRP='3RD PAGES FOR*DESTRUCTIVE*READ REQUESTED'
QB3TWFD ='3RD WORKFILES*DENIED*DURING SORT/MERGE'
QB3TWFF ='3RD SORT/MERGE*INEFFICIENT*(BUFFER SHORTAGE)'
QB3TWFM ='3RD MAX WORKFILES*ALLOCATED*DURING SORT/MERGE'
QB3TWFR ='3RD REQUESTS TO*QUERY FOR WORKFILES*IN SORT/MERGE'
QB3TWFT ='3RD WORKFILES*REQUESTED*DURING SORT/MERGE'
QB4TLPF ='4TH LIST*PREFETCHES*REQUESTED'
QB4TMAX ='4TH BUFFERPOOL*NOT SUPPORT*CONCURR WORKFILES'
QB4TWBVQ='4TH PAGES*DEQUEUED FOR*DESTRUCT READ'
QB4TWDRP='4TH PAGES FOR*DESTRUCTIVE*READ REQUESTED'
QB4TWFD ='4TH WORKFILES*DENIED*DURING SORT/MERGE'
QB4TWFF ='4TH SORT/MERGE*INEFFICIENT*(BUFFER SHORTAGE)'
QB4TWFM ='4TH MAX WORKFILES*ALLOCATED*DURING SORT/MERGE'
QB4TWFR ='4TH REQUESTS TO*QUERY FOR WORKFILES*IN SORT/MERGE'
QB4TWFT ='4TH WORKFILES*REQUESTED*DURING SORT/MERGE'
QIST - No Change.
QTXA - No Change.
EDM - New Variables (4):
QISEKTG ='REQ*FOR PT*SECTIONS'
QISEKTL ='LOAD PT*SECT FROM DASD'
QISEKT ='PAGES*USED FOR PT'
QISESKPT='PAGES*USED FOR SKPT'
i. DB2ACCT - Type 101 SMF Record changes:
See previous discussion on matching DB2 transactions to CICSTRAN
transactions. The new QWHCTOKN variable is used to create
NETSNAME='ORIGINATING*SYSTEM*NET-NAME'
UOWTIME ='UNIT-OF-WORK*INTERNAL*TIME STAMP'
which are used to match CICS and DB2 transactions.
QWAC - New Variables(9):
QWACALCT='ARCHIVES*LOG MODE*(QUIESCE)'
QWACAWTR='WAIT TIME FOR READ I/O UNDER ANOTHER THREAD'
QWACAWTW='WAIT TIME FOR WRITE I/O UNDER ANOTHER THREAD'
QWACAWTE='WAIT TIME DUE TO SYNCHRONOUS EXEC UNIT SWITCH'
QWACALOG='WAIT TIME DUE TO PROCESSING OF ARCHIVE LOG'
QWACARNL='WAITS FOR LOCK/LATCH'
QWACARNR='WAITS FOR READ I/O UNDER ANOTHER THREAD'
QWACARNW='WAITS FOR WRITE I/O UNDER ANOTHER THREAD'
QWACARNS='WAIT TRACE EVENTS PROCESSED FOR SYNC EXEC UNIT SW'
QXST - New Variables (4):
QXSETHV ='SET*HOST-VARIABLE*STATEMENTS'
QXALDAB ='ALTER*DATABASE*STATEMENTS'
QXDRPPKG='DROP*PACKAGE*STATEMENTS'
QXDSCRTB='DESCRIBE*TABLE*STATEMENTS'
QBAC - New Variables(4):
QB1CLPF ='1ST LIST*PREFETCHES*REQUESTED'
QB2CLPF ='2ND LIST*PREFETCHES*REQUESTED'
QB3CLPF ='3RD LIST*PREFETCHES*REQUESTED'
QB4CLPF ='4TH LIST*PREFETCHES*REQUESTED'
QTXA - No Change.
QLAC - New Variables(10):
QLACBRBF='BLOCKS RECEIVED USING BLOCK FETCH'
QLACBROW='ROWS IN MSG BUFFER IF BLOCK FETCH'
QLACBTBF='BLOCKS TRANSMITTED USING BLOCK FETCH'
QLACCBLB='SWITCHES*FROM CONT BLK MODE TO LIM BLK'
QLACCIEL='LARGEST*(QLACCNVA-QLACCNVT)*VALUE'
QLACCNVA='NUMBER OF SUCCESSFUL ALLOCATIONS'
QLACCNVT='CONVERSATIONS*TERMINATED'
QLACFLG1='PRIVATE*PROTOCOLS*USED?'
QLACFLG2='DRDA*PROTOCOLS*USED?'
QLACRBND='SQL STMTS*BOUND FOR*REMOTE ACCESS'
j. T102Snnn - Type 102 IFCID nnn SMF Record changes:
IFCID 29 New Variables:
QW0029CI='COLLECTION*ID'
QW0029CT='CONSISTENCY*TOKEN'
QW0029GL='LENGTH*OF THE PT*SECTION'
QW0029GN='SEQ NR*WITHINRDS*SECTION'
QW0029KN='RDS*IDENTIFICATION*NUMBER'
QW0029LN='LOCATION*NAME'
QW0029PI='PACKAGE*ID'
QW0029RS='RESERVED*QW0029RS'
QW0029SV='SERVICEABILITY*QW0029SV'
IFCID 30 New Variables:
QW0030CI='COLLECTION*ID'
QW0030CT='CONSISTENCY*TOKEN'
QW0030GL='CALLS TO*DATA MANAGER*FOR THE PT'
QW0030GN='SEQ NR*WITHINRDS*SECTION'
QW0030KN='RDS*IDENTIFICATION*NUMBER'
QW0030LN='LOCATION*NAME'
QW0030PI='PACKAGE*ID'
QW0030SV='SERVICEABILITY*QW0030SV'
IFCID 31 New Variables:
QW0031CI='COLLECTION*ID'
QW0031CT='CONSISTENCY*TOKEN'
QW0031GL='LENGTH*OF THE PT*SECTION'
QW0031GN='SEQ NR*WITHINRDS*SECTION'
QW0031KN='RDS*IDENTIFICATION*NUMBER'
QW0031LN='LOCATION*NAME'
QW0031PI='PACKAGE*ID'
QW0031SV='SERVICEABILITY*QW0031SV'
IFCIDs 53, 58-61, 64-66:
Package Name (described previously) was added, and program name
was changed from 8 to 18 bytes.
IFCIDs 81, QW0081RQ was increased from $1 to @2 bytes.
IFCIDs 93, QW0081RB was increased from $40 to $48 bytes.
IFCID 106 New variables:
QWP1FLAG='FLAG*BITS'
QWP3MQP ='MAXIMUM*QUIESCE*PERIOD'
QWP4CNTL='CONTROL*PARAMETER'
QWP4KSIZ='CONTROL*PACKAGE*HASH TABLE5'
QWP4MDDN='MAX NR*OF DDS*WITH HOLD'
QWP4REGA='DDL REG*ART*NAME'
QWP4REGC='DDL REG*TABLE*OWNER'
QWP4REGF='DDL REGISTRATION*FACILITY*FLAG'
QWP4REGN='DDL REG*DATABASE*NAME'
QWP4REGO='DDL REG*ORT*NAME'
QWP4SIT ='SITE*TYPE*FLAG'
QWP4TDDN='VALUE FOR*TRIGGER*DRAIN'
IFCID 108 New variables:
QW0108OW='SPECIFIED OWNER*OF PLAN*OR PACKAGE'
QW0108TY='TYPE OF*OBJECT BOUND*OR REBOUND'
QW0108QL='QUALIFIER FOR*UNQUALIFIED*OBJECTS'
QW0108CA='AUTHORIZATION*CACHESIZE'
QW0108NL='LOCATION*OF*PACKAGE'
QW0108NC='COLLECTION*ID OF*PACKAGE'
QW0108NI='PACKAGE*ID'
QW0108NT='CONSISTENCY*TOKEN*OF PACKAGE'
QW0108VL='VERSION*LENGTH'
QW0108VN='VERSION*NAME'
IFCID 110 New variables:
QW0110TY='TYPE OF*OBJECT*FREED'
QW0110PL='LOCATION*OF*PACKAGE'
QW0110PC='COLLECTION*ID'
QW0110PI='PACKAGE*ID'
QW0110PT='CONSISTENCY*TOKEN'
QW0110VL='VERSION*LENGTH'
QW0110VN='VERSION*NAME'
IFCID 124 new variables:
QW0124CI='COLLECTION*NAME'
QW0124PN='PACKAGE*ID'
QW0124CN='CONSISTENCY*TOKEN'
QW0124NI='NETWORK*ID'
QW0124LM='LUNAME'
QW0124UV='UNIQUENESS*VALUE'
QW0124CC='COMMIT*COUNT'
IFCID 145 new variables:
QW0145LN='LOCATION*NAME'
QW0145PC='PACKAGE*COLLECTION*ID'
QW0145PN='PROGRAM*NAME'
IFCID 141 QW0141CO increased from 2 to 4 bytes.
IFCID 147, 148 new variables:
QW0148LN='LOCATION*NAME'
QW0148CI='COLLECTION*NAME'
QW0148PN='PACKAGE*ID'
QW0148CN='CONSISTENCY*TOKEN'
QW0148NI='NETWORK*ID'
QW0148LM='LUNAME'
QW0148UV='UNIQUENESS*VALUE'
QW0148CC='COMMIT*COUNT'
QW0148EB='WAIT FOR DB2 SERVICE TASK BEGIN'
QW0148EE='WAIT FOR DB2 SERVICE TASK END'
QW0148RB='WAIT FOR ARCHIVE LOG MODE(QUIESCE) BEGIN'
QW0148RE='WAIT FOR ARCHIVE LOG MODE(QUIESCE) END'
QW0148CT='CONVERSATION*TYPE*INDICATOR'
2. Using MXG DB2 members ANALDB2R, ANALDBTR, and READDB2
Chuck Hopf, Hopf Consulting, (214) 327-0881
Extracting and reporting on the performance data recorded by
DB2 can be accomplished quickly and easily using SAS and
Merrill's Expanded Guide (MXG.) This paper documents the use of
these software packages in analyzing the performance of DB2
systems.
INTRODUCTION
Since the introduction of DB2, SMF data has been available to assist the
Performance Analyst and the DB2 DBA (Data Base Administrator) in problem
determination and planning for the use of DB2. MXG has supported DB2
SMF data since availability, and provides a reporting facility that
produces reports similar to IBM's DB2PM Performance Monitor product.
This facility has the capacity to extract data from live SMF datasets
(SYS1.MANx), SMF dump datasets, or in some instances, GTF datasets. DB2
data should always be directed to SMF instead of GTF for several
reasons. First and foremost, the "write" of DB2 data to GTF requires an
actual physical I/O, managed by DB2, which can interfere with that which
is being measured, whereas the "write" of DB2 data to SMF is not a
physical write, but rather is a data movement at memory speed from the
DB2 address space to the SMF address space, which increases the accuracy
of DB2 timestamps. Second, SMF records are 32760 bytes long, whereas
GTF is limited to 256 bytes in each physical record, which means that
not only is the overhead of creation significantly less with SMF than
with GTF, but also your processing programs will be more efficient using
SMF data. Finally, if the logical data length exceeds 256 bytes to GTF,
a very non-standard spanning format is used, in which actual data fields
can be split across records, and this format is not supported by SAS;
thus some DB2 GTF records cannot be read with SAS!
MXG processes DB2 records and will also store DB2 data as SAS datasets
for future reference or for additional reporting without reprocessing
SMF data. The stored detail data can also be TRENDed for developing
long term capacity trends and plans of DB2 performance and resources.
MXG uses the "old style, substitution" MACRO definitions as shorthand
for the processing of all SMF data, and uses the "new" %MACRO facility
for report generation and selection. The primary reporting %MACRO in
MXG is %ANALDB2R, which itself invokes the %MACRO named %READDB2 (if
raw SMF data is to be read), which then dynamically constructs a SAS
program of old style _VAR and _CDE macro definitions (that are defined
in ANALDBTR) to create the needed SAS datasets for DB2 reporting.
DB2 Statistics records cannot be used directly, because they contain the
cumulative total in their counters, rather than delta values for the
interval. This requires the SAS datasets to be created, then sorted,
and then de-accumulated; member DIFFDB2 uses the powerful DIF() function
to accomplish this deaccumulation.
In the case of many types of TRACE data, such as a READ event, a record
is written at the beginning of an event and a second record is written
at the end of the event. These records must be paired to measure the
true results of the operation. The first record contains information on
what was requested while the second the records the activity counts.
Member ANALDBTR provides the "PAIRing" logic for DB2 Trace records.
READING SMF DATA
Several options are available for reading the SMF or GTF data using MXG.
With three SMF records and over 200 subtypes, processing all possible
DB2 records can require appreciable virtual storage. Under SAS 5.18 it
is not possible to process all DB2 2.3 records in a single pass, but the
SAS Version 6 implementation eliminated this constraint, although it may
require a MEMSIZE=28M option in your CONFIG member.
A normal MXG program to process SMF data will %INCLUDE a member of the
MXG source library, named TYPExxxx, to build a SAS dataset TYPExxxx.
The TYPExxxx member invokes three MXG substitution style macros named
_SMF, _VARxxxx, and CDExxxx that are defined in member VMACxxxx. (All
MXG old style macros begin with an underscore as a naming convention.)
Figure 1 is an example for the SMF type zero (IPL) record processing.
//SASJOB JOB ACCOUNTING-INFO
//SAS EXEC SAS606,CONFIG='MXG.SOURCLIB(CONFIG)'
//SOURCLIB DD DSN=USERID.SOURCLIB,DISP=SHR
// DD DSN=MXG.SOURCLIB,DISP=SHR
//LIBRARY DD DSN=MXG.FMTS606,DISP=SHR
//SMF DD DSN=MY.SMFDATA,DISP=SHR
//SYSIN DD *
%INCLUDE SOURCLIB(TYPE0);
Member TYPE0 of the MXG SOURCLIB would contain:
%INCLUDE SOURCLIB(VMACSMF,VMAC0);
DATA
_VAR0
_SMF
_CDE0
;
Figure 1. Example of typical MXG job.
MXG's TYPEDB2 member is similar to the above example, but it processes
SMF type 100 records (subtypes 0 and 1) and SMF type 101 records, using
the _VARDB2 and _CDEDB2 macros defined in VMACDB2 to simultaneously
creates the three DB2 datasets DB2ACCT, DB2STAT0, and DB2STAT1. It also
then invokes member DIFFDB2 to deaccumulate DB2STAT0 and DB2STAT1.
The _VARxxxx and _CDExxxx "record-level" macros process one or more SMF
records, but for DB2 trace data, additional "subtype-level" macros are
defined in VMAC102 that permit subtype selectivity for each IFCID
(subtype) that can be created. Defined in member VMAC102, they have
names of the form _V102nnn and _C102nnn, where nnn is the IFCID to be
processed. Just as _VARxxxx and _CDExxxx macros can be combined to
create multiple datasets from multiple records simultaneously, the
_V102nnn and _C102nnn macros can be used for multiple subtype processing
in a single run. Macros _VAR102 and _CDE102 are defined in VMAC102, and
combine all possible "subtype-level" _V102nnn/_C102nnn macros to
simultaneously build all 125 possible trace datasets, and member
TYPE102 looks just like TYPE0 example in Figure 1.
Let's examine the wrong and the right way to use MXG for DB2.
One way to read the DB2 Accounting, Statistics, and Trace Data is shown
in Figure 2, complete with sample JCL for SAS 6.06.
//SASJOB JOB WHAT_WORKS
//SAS EXEC SAS606,CONFIG='MXG.SOURCLIB(CONFIG)'
//SOURCLIB DD DSN=USERID.SOURCLIB,DISP=SHR
// DD DSN=MXG.SOURCLIB,DISP=SHR
//LIBRARY DD DSN=MXG.FMTS606,DISP=SHR
//SMF DD DSN=MY.SMFDATA,DISP=SHR
//SYSIN DD *
%INCLUDE SOURCLIB(TYPEDB2,TYPE102);
Figure 2: One way to READ DB2 SMF DATA
THIS IS THE WRONG WAY. Concatenation of the two TYPExxxx members will
build all possible datasets, but each TYPExxxx member will read the
SMF dataset one time. In addition, all of the datasets built will have
been written to the work file (a temporary dataset), and nothing will be
stored! You could add a permanent DD (perhaps //PDB DD ...) and then
PROC COPY IN=WORK OUT=PDB to store the datasets, but it would still need
two passes of the entire SMF file for your DB2 data, and do you really
want all 195 subtypes of the 102 record?
A better way is to tell MXG what specific datasets/subtypes you want by
%INCLUDEing the macro definitions, and then specifying what you want:
//SYSIN DD *
%INCLUDE SOURCLIB(VMACDB2,VMAC102,VMACSMF,IMACKEEP);
DATA
_VARDB2
_V102106
_V102038
_SMF
_CDEDB2
_HDR102
_C102106
_C102038
_END102
%INCLUDE SOURCLIB(DIFFDB2);
Figure 3: Reading Specific IFCIDs
As many IFCIDs as desired may be specified in a single program when
running SAS version 6.06. Under SAS version 5.18, multiple IFCIDs may
be read, but the number is limited based on the number of iterative DO
loops contained in the code to read each IFCID. Since this number varies
from one IFCID to another, it is not possible to predict how many IFCIDs
can be read with SAS 5.18. The SAS program in figure 3 will read the
DB2 accounting, statistics, system parameter (IFCID 106) and the IFCID
38 records. DIFFDB2 is then invoked to finish the process by sorting
the accounting records and deaccumulating the statistics records, but it
still does not store any datasets nor produce any reports. Read on!
DB2 traces can also be invoked at the Trace Class level by the type of
Trace Class (Accounting, Audit, Performance, Statistics, etc.) and the
Trace Class Number. Thus MXG also provides Trace Class macros.
MXG supports the ACcounting, AUdit, GLobal, MOnitor, and PErformance
trace classes. For each class, there is a _VTRccmm and a _CTRccmm macro
corresponding to the "subtype-level" but creating all possible datasets
from a trace class instead of one subtype. The "cc" is the trace class,
the first two characters of the trace name ("AC" for ACcounting, "AU"
for AUdit, etc.), and the "mm" is the trace class number. PErformance
Trace Class 3 macros are named _VTRPE03 and _CTRPE03. See Figure 4.
//SYSIN DD *
%INCLUDE SOURCLIB(VMACDB2,VMAC102,VMACSMF);
DATA
_VARDB2
_VTRPE03
_VTRMO01
_SMF
_CTRDB2
_HDR102
_CTRPE03
_CTRMO01
_END102
%INCLUDE SOURCLIB(DIFFDB2);
Figure 4: Reading Specific Trace Classes
This is still not the best way, because when using this technique, it is
your responsibility to ensure that no duplicate IFCIDS are generated by
the multiple trace classes specified. For example, Monitor Trace Class
3 and Performance Trace Class 2 both generate IFCIDs 174 and 175. If you
used _VTRMO03 and _VTRPE02 in the same program, a syntax error occurs.
To assist the user in resolving this problem and simplify the process of
reading DB2 trace data, a %MACRO was constructed to perform the tasks
of determining the IFCIDs needed for any trace class and to generate the
code while preventing the possibility of duplicate IFCID code. The new
macro is called %READDB2, and it IS the best way to process DB2 data.
%READDB2 lets you enter lists of IFCIDs and/or Trace Classes, and it
generates the SAS code, eliminating any duplicate IFCID processing you
might have requested. There are four macro arguments defined which let
you modify the %READDB2 processing. They are listed below in Table I,
and are more completely defined in the member READDB2 itself.
Table I: READDB2 Parameters and Meanings
Parameter Default/Meaning
INFILE SMF - May be either SMF or GTF depending on the source
of the DB2 data records to be read.
PDBOUT PDB - Default DDNAME describing the SAS data library
into which the DB2 datasets will be written.
WORK DDNAME is used if PDBOUT is blank.
IFCIDS blank - The list of IFCIDS to be read. May be any
combination of the words ACCOUNTS, STATISTICS,
ALL, or any number from 0 to 199.
TRACECLS blank - The DB2 Trace class(es) to be read. A list of
previously described trace class names.
INVOKEBY Used for communication with ANALDB2R. DO NOT MODIFY.
The invocation of %READDB2 to read the Accounting and Statistics records
and the System Parameter Record (IFCID 106) is shown in Figure 5.
//SYSIN DD *
%READDB2(INFILE=SMF,IFCID=ACCOUNT 106);
Figure 5: Invoking %READDB2
This invocation of %READDB2 would read all of the accounting, statistics
and IFCID 106 records in the input SMF file, invoke DIFFDB2 to sort the
accounting records and deaccumulate the statistics records, and copy all
of the resulting datasets to the ddname PDB.
Much more complex invocations of READDB2 are possible. When running
Version 6 of SAS, it is possible to read all DB2 created records in a
single pass of the SMF data using READDB2 as illustrated in figure 6.
//SYSIN DD *
%READDB2(INFILE=SMF,IFCIDS=ALL);
Figure 6: Reading ALL DB2 Record Types
Selection of multiple IFCIDS and TRACE classes is also possible. Figure
7 illustrates the invocation to read IFCIDS, 34 35 37 38 and the Audit
class 1 and 2 data in a single pass. READDB2 dynamically determines if
duplicate IFCIDs are called for in a request and eliminates the
duplicate requests prior to generating the SAS program to read the data.
//SYSIN DD *
%READDB2(
%READDB2(IFCIDS=34 35 37 38,TRACECLS=AU01 AU02);
Figure 7: Reading Multiple IFCIDs and Trace Classes with READDB2
%READDB2 will execute under both version 5 and 6 of SAS. Under Version
5 a region size of 6M is recommended, but a message is generated that a
344 ERROR is possible if too many IFCIDs were requested. If this does
occur, reduce the number of IFCIDs or trace classes requested and rerun
the job. If ALL is specified as the IFCID when running version 5 of
SAS, the program is automatically aborted since it is not possible to
run (or even compile) a program with ALL IFCIDs under Version 5 of SAS>
For version 6 users, a MEMSIZE of 28M is required to process ALL records
in a single pass. In most other instances the default MXG MEMSIZE of 24M
is adequate.
"DIFFing and PAIRing" DB2 Data
As mentioned earlier, some types of DB2 data require further processing
before reports can be generated.
The Statistics data is maintained as running counters in the SMF data
and any given record reflects the sum of all activity for that
particular execution of a DB2 subsystem up to the point in time at which
the SMF record was written. For this reason, any time that TYPEDB2 is
invoked, in BUILDPDB processing, or READDB2 processing, member DIFFDB2
is included to perform the DIFFing of the statistics and to sort the
accounting data into a somewhat meaningful sequence.
The DB2 Trace records do not suffer from this particular problem, but do
require preprocessing before reporting. In many instances, the data
required to report on any specific DB2 event (READ, BUFFER GET, SQL
Statements, etc.) are contained in two records. The first record
typically contains information describing the event such as the Database
accessed, the PAGESET, the SQL statement type to be executed, etc. The
second record contains the information about how the request was
processed. Was it successful, how much I/O was performed, how many rows
were read, etc. Since many other events (including others of the same
type) may intervene between the start and end of a particular event,
putting these records back together so that a report can be generated is
not exactly straightforward.
Member ANALDBTR contains the SAS code in the form of substitution macros
needed to "PAIR" the current DB2 record pairs. Figure 8 is the SAS code
needed to read the DB2 trace record subtypes 6 and 7 (Begin and End Read
IO) using READDB2 to read the data and then invoking the statement style
_006PAIR macro to perform the "PAIRing." Each of these macros creates
one or more datasets where the dataset name is SxxxSyyy where xxx is the
Begin Event subtype and yyy is the End Event subtype. (The subtype 183
is paired with itself, and creates I183R183, the only naming exception).
//SYSIN DD *
%READDB2(IFCIDS=6 7);
%INCLUDE SOURCLIB(ANALDBTR);
_006PAIR
Figure 8: Reading and Pairing DB2 Trace Records
Multiple _xxxPAIR macros can be invoked in a single pass. Figure 9
illustrates the same technique to create the PAIRs for Read (S006S007)
Write (S008S009 S010S009) and SQL statements (S059S058 S060S058 S061S058
S062S058 S063S058 S064S058 S065S058 S066S058). Notice that in the case
of both the write and SQL activity that there are multiple pairs that
share a common end record (9 for write and 58 for SQL.)
//SYSIN DD *
%READDB2(IFCIDS=6 7 8 9 10 58 59 60 61 62 63 64 65 66);
%INCLUDE SOURCLIB(ANALDBTR);
_006PAIR
_008PAIR
_058PAIR
Figure 9: Processing Multiple Pairs
Another method of creating multiple record pairs is with the ANALDBTR
new style macro. This macro will generate the calls to the requested
_xxxPAIR macros based on parameter driven input. The parameters for
ANALDBTR with their default values are contained in table II.
Figure 10 illustrates the use of ANALDBTR to create the same paired data
(Read, Write, and SQL) as figure 9.
//SYSIN DD *
%READDB2(IFCIDS=6 7 8 9 10 58 59 60 61 62 63 64 65 66);
%ANALDBTR(PAIRS=6 8 58);
Figure 10: Using ANALDBTR to Pair DB2 Trace Records
ANALDBTR could be used to produce simple reports of I/O activity from
the PDB (assuming that the appropriate data is in the PDB DDNAME) simply
by pairing the I/O related records and PROC PRINTing the resulting
datasets as illustrated by Figure 11. _PAGE_ 27
%READDB2(IFCIDS=6 7 8 9 10);
%ANALDBTR(PAIRS=6 8);
PROC PRINT DATA=S006S007 SPLIT='*'; TITLE 'DB2 READ I/O';
PROC PRINT DATA=S008S009 SPLIT='*'; TITLE 'DB2 SYNC WRITE I/O';
PROC PRINT DATA=S010S009 SPLIT='*'; TITLE 'DB2 ASYNC WRITE I/O';
Figure 11. Simple I/O Reports if data is in your PDB
These would be very simplistic reports but might be very useful and
quick. For more sophisticated DB2 reporting requirements, MXG provides
a more robust set of reports.
Producing DB2 Performance Reports
Production of DB2 performance reports under MXG is accomplished by the
%ANALDB2R macro defined in member ANALDB2R. This member of the MXG
SOURCLIB contains many SAS MACROs (both %MACROs and substitution style)
that work together to allow dynamic selection of the types and sort
sequences of many of the DB2 reports. The MXG implementation requires
certain SAS options to be specified. Under Version 5, these required
options are set by %INCLUDE SOURCLIB(SASOPTV5); as the first statement
of your program, with OPTIONS='MWORK=28K' specified on your // EXEC SAS
statement. Under SAS Version 6, the CONFIG= JCL parameter points to the
CONFIG member which specifies these required options.
By default, ANALDB2R produces the Accounting Summary, Accounting Detail,
Statistics Summary, and the System Parameter reports. As with all DB2
reports using MXG, the report is only produced if the data required for
the report is available. The data required for each class of DB2 report
was contained in Table I (see page 27, above). In many cases, not all
of the listed data is mandatory to produce the report.
Refer to the DB2PM Reference Manual SH20-6858-02 for report contents.
%READDB2 is used by %ANALDB2R to read the raw SMF data if the data is
not already contained in the PDB. Additionally, raw SMF data and data
from one or more PDB datasets can be combined for reporting so that both
old and current measurements may be combined on a single report. The
only restriction on this is that SMF (or GTF) must be the first entry in
the PDB= list.
PDBOUT can be used to store the resulting SAS datasets in any desired
SAS data library simply by specifying PDBOUT=DDNAME, where DDNAME is the
DDNAME describing the SAS data library.
Figure 12 is an example of JCL to read the DB2 data using ANALDB2R
including performance trace classes 1 and 2 and audit class 1 and place
the resulting data in the PDB before creating the default reports.
//SYSIN DD *
%ANALDB2R(PDB=SMF,PDBOUT=PDB,TRACECLS=PE01 PE02 AU01);
Figure 12: JCL to read data with ANALDB2R
When the TRACECLS or IFCID options of %ANALDB2R are used, those datasets
are added to the list of IFCIDs processed by default. Thus this example
also produces the accounting, statistics, and system parameter datasets
because they are enabled by default in ANALDB2R.
Figure 13 is an example of creating only the Accounting Summary report
while reading raw SMF data and combining the results with data from a
PDB and a TREND database. The specification to eliminate a default
report is PMxxxnn=NO where xxx is the report type and nn is the report
number.
//SYSIN DD *
%ANALDB2R(PDB=SMF PDB TREND,PMACC02=NO,PMSTA02=NO,PMSPR01=NO);
Figure 13: Accounting Summary Report from SMF, PDB, and TREND
OTHER REPORTS and FEATURES
A %MACRO variable exists for each report defined in %ANALDB2R, which has
an initial value of either blank or YES. To enable a report which has
default NO, it is only necessary to specify the MACRO variable name
with a "YES" in the form MACRO Variable = YES at the time ANALDB2R is
begun. Table IV identifies all of the reports available the MACRO
variable name, and the default value used, Table III the IFCIDs required
to produce each report, and Table II, the Trace Classes which should be
enabled by the DBA to produce the data for a specific report. Figure 14
is an example requesting the reading of raw SMF data from the SYS1.MAN1
dataset, suppressing the default reports, and creating the Lock
Suspension Summary report.
//SYSIN DD *
%ANALDB2R(
PDB=SMF,
PMACC01=NO, /* NO ACCOUNTING SUMMARY */
PMACC02=NO, /* NO ACCOUNTING DETAIL */
PMAUD01=YES, /* PRINT LOCK SUSPENSION SUMMARY */
PMSTA02=NO, /* NO STATISTICS SUMMARY REPORT */
PMSPR01=NO); /* NO SYSTEM PARAMETER REPORT */
Figure 14: Read SMF and Produce Only the Lock Suspension Report
To reduce the volume of data and reports that must be processed by the
analysts, many variables are also supplied for data selection. In each
case, a MACRO variable name must be supplied followed by the "= string "
where the string is the value(s) desired. The length of the string
supplied as the MACRO value is also used to determine the length of the
comparison. Thus, if PLAN=MXG were specified, all plans beginning with
the string MXG would be selected.
To further simplify the analysis process, accounting and audit reports
may be sorted by up to three different variables and the statistics
report can be summarized to intervals other than the intervals at which
the data was written.
For example, if a DB2 plan that was known to have a performance problem
was being executed in a production system while the "new improved
version" of the same plan was being executed in a test system, the
Accounting Summary report could be run specifying "SORTBY=PLAN DB2".
This would result in the data for the two subsystems appearing on two
consecutive lines of the same report simplifying the task of comparing
the performance results.
As another example, the Statistics Detail Report can be a very valuable
tool, but the volume of the report can be quite large. Each interval
can generate up to nine pages, depending on the number of buffer pools
in use. With four buffer pools, each hour could potentially result in
36 pages of SYSOUT for the analyst to digest. Specification of
"INTERVAL=HOUR" would summarize the data at the hourly level while
"INTERVAL=DATE" would result in summarization at a daily level.
The Audit Detail and Trace reports also may generate large volumes of
data. To reduce this volume, the AUDIT= parameter is provided. From one
to seven of the Audit classes may be specified separated by blanks and
terminated by either a comma or a left parenthesis (the parenthesis is
used if it is the last parameter and the comma in all other cases.) Thus
the specification of "AUDIT=AUTHFAIL AUTHCNTL" used with either PMAUD02
or PMAUD03 would result in the creation of the Audit reports for
Authorization Failures or Authorization Control events.
Data may also be reduced through the selection of a time range for the
report. This is accomplished with the BEGTIME and ENDTIME parameters.
Either or both of these parameters may be used employing the SAS
DATETIME format to describe the desired date and time. To select all
data beginning with August 8, 1990, specify "BEGTIME=08AUG90:00:00:00".
Note that quotation marks around the DATETIME literal are not required.
A complete list of the %ANALDB2R parameters and their meanings is
contained in Table V, (page 32, below), and in member ANALDB2R itself.
EXAMPLES OF USE
The following examples all make the assumption that the required data
for the reports has been gathered and placed in the MXG PDB. Figure 15
presents the JCL required to execute these examples. For example
purposes, SAS version 6.06 and MXG version 9 are assumed.
//SASJOB JOB ACOUNTING-INFO
//SAS EXEC SAS606,REGION=4M,
// CONFIG='MXG.SOURCLIB(CONFIG)'
//SOURCLIB DD DSN=USERID.SOURCLIB,DISP=SHR
// DD DSN=MXG.SOURCLIB,DISP=SHR
//LIBRARY DD DSN=MXG.FMTS606,DISP=SHR
//PDB DD DSN=MXG.PDB,DISP=SHR
//SYSIN DD *
Figure 15 Example JCL
Example 1:
Produce the Accounting Detail and Accounting Summary reports sorted by
DB2 subsystem and plan. Also produce the Statistics Detail report
summarized at one hour intervals. The SYSIN required is contained in
Figure 16.
Notice that the default reports that are not desired must be specified
with a "NO" to suppress the printing of the reports.
//SYSIN DD *
%ANALDB2R(PMACC01=YES,PMACC02=YES,PMSTA01=YES,PMSTA02=NO,
PMSPR01=NO,SORTBY=DB2 PLAN,INTERVAL=HOUR);
Figure 16: Example 1
Example 2:
Produce all default reports in the default sequences and also produce
the Accounting Summary Report sorted by plan, connection, and subsystem.
%ANALDB2R;
%ANALDB2R(PMACC01=NO,PMACC02=YES,PMSTA02=NO,PMSPR01=NO,
SORTBY=PLAN CONNID DB2);
Figure 17: Example 2
In this example it was necessary to execute ANALDB2R twice since only a
single combination of SORTBY values may be specified. Also notice that
the parameters may be placed on a single line so long as the parameters
(not their values) are separated by commas.
Example 3:
Produce all default reports and produce the Audit Detail Report for
authorization failures and audited DDL accesses. Also produce the Lock
Suspension Summary and the Buffer Pool Manager Summary.
In this example, the entire command sequence is entered as a continuous
string. Notice that the parameters may be specified in any order and
that the number of spaces is not significant.
%ANALDB2R(PMAUD02=YES,AUDIT=AUTHFAIL DDL,PMLOK01=YES,PMIOS01=YES);
Figure 18: Example 3
EXECUTION NOTES ON THE SAS LOG
ANALDB2R prints a list of the parameters entered when executed.
MXGNOTES and MXGWARN messages may be produced by ANALDB2R. The most
important are those indicating that either datasets could not be found
or that although the datasets exist, no data could be found that met the
selection criteria specified by the user. For example, the Lock
Suspension Report requires the S044S045 and the T102S054 datasets. If
none of the datasets exist in the PDB DDNAME, or they all contain zero
observations, a MXGWARN is produced.
If a CLIST has been installed with the necessary libraries defined
(SOURCLIB, SASLIB, and PDB), ANALDB2R can also be executed from a SAS
TSO session. This is not recommended unless a 132 character screen is
available and all of the data required has already been placed in SAS
datasets. All of the reports generated by ANALDB2R are more than 80
characters wide and the CPU time to reduce the SMF data may be
prohibitive for a TSO session (because of higher overhead vs batch).
Table II: Trace Classes Required to Produce Reports
Type of Report Trace Type Trace Class
Accounting Accounting 1 2 3
Audit Audit 1 2 3 4 5 6 7 8 9
IO Subsystem Performance 4 5
Locking Performance 6
SQL Trace Accounting 1 2 3
SQL Trace Performance 1 2 3 4 6 8 13
Statistics Statistics 1
System Parameter System Parameter Any
Record Trace ALL ALL
Transit Time Performance ALL
Table III: IFCIDs Required for Reports
Report Type IFCIDs
Accounting 3
Audit 23 24 25 55 83 87 105 107 140-146
IO Subsystem 6-10 29-30 34-41 105 107 114-116 119-120
Locking 44 45 54 105 107
SQL Trace 3 6-9 11-12 15-20 22 44-45 53 55 58-66 68-71 73-75
86-89 95-96 105 107 124-125 183
Statistics 1 2
System Parms 106
Record Trace ALL
Transit Time 4-12 19-20 22-25 44-45 53 55 58-66 68-79 84-92 95-97
105 107-111
Table IV - Available Reports and Defaults
DB2 Report Name MACRO DEFAULT
Accounting Summary Report PMACC01 YES
Accounting Detail Report PMACC02 YES
Accounting Trace Short Form PMACC03
Accounting Trace Long Form PMACC04
Audit Summary Report PMAUD01
Audit Detail Report PMAUD02
Audit Trace Report PMAUD03
Buffer Pool Manager Summary PMIOS01
EDM Pool Manager Summary PMIOS02
LOG Active LOG Report PMIOS03
ARCHIVE LOG/BSDS Report PMIOS04
Lock Suspension Summary PMLOK01
Lock Contention Summary PMLOK02
Lock Contention Trace PMLOK03
SQL Trace Summary PMSQL01
SQL Short Trace Report PMSQL02
SQL Long Trace Report PMSQL03
SQL DML Trace Report PMSQL04
Statistics Detail Report PMSTA01
Statistics Summary Report PMSTA02 YES
Statistics Trace Long Form PMSTA03
Statistics Trace Short Form PMSTA04
System Parameter Report PMSPR01 YES
Record Trace Short Form PMTRC01
Record Trace Long Form PMTRC02
Transit Time Report PMTRN01
Table V - Selection Parameters
Parameter Description
DB2 A list of full or partial DB2 subsystems to
be selected.
PLAN A list of full or partial DB2 Plans to be
selected.
CONNID A list of full or partial CONNECTION IDs to
be selected.
CORRID A list of full or partial CORRELATION IDs to
be selected. (See MXG member IMACDB2)
AUTHID A list of full or partial Authorization IDs
to be selected.
DATABASE A list of database IDs (HEX string) to be
selected.
PAGESET A list of pageset IDs (HEX string) to be
selected.
DB2LOCAL A list of the LOCAL DB2 systems in a DDR
environment to be selected.
DB2REMOT A list of the REMOTE DB2 systems in a DDR
environment to be selected.
TRACENUM A list of the DB2 TRACE numbers to be selected
NETSNAME A list of the NETSNAMEs to be selected. *
NETID A list of the QWHSNIDs to be selected. *
LUNAME A list of the QWHSLUNMs to be selected. *
INSTANCE A list of the QWHSLUUVs to be selected. *
TOKEN A list of the QWHCTOKNs to be selected. *
CNETID A list of the QWACNIDs to be selected. *
DNETID A list of the QWHDNETIs to be selected. *
DLUNAME A list of the QWHDLUNMs to be selected. *
DINSTNCE A list of the QWHDUNIQs to be selected. *
SORTBY A list of up to three variables from the
above by which the data should be sorted.+
BEGTIME The earliest DATETIME that will be selected
from the specified input data source.
ENDTIME The latest DATETIME that will be selected
from the specified input data source.
AUDIT Up to seven AUDIT classes to be selected.
Valid values are:
AUTHFAIL - Authorization Failure
AUTHCNTL - Authorization Control
DDL - Audited DDL Access
DML - Audited DML Access
BIND - DML at BIND
AUTHCHG - Authorization BIND
UTILITY - Utility Access
INTERVAL Summarize the data to the specified interval.
(See VMXGSUM for documentation)
MYTIME User specification of time interval. See
VMXGSUM for details.
* version 2.3 of DB2 only
+ Valid only for the PMACC01, PMACC02, PMAUD01, and
PMAUD02 reports.
V. SAS Technical Notes.
1. CALL YOUR SAS SALESPERSON AND REQUEST EARLY SHIPMENT OF SAS 6.07!!!
SAS 6.07 has begun shipping to all their customers, but it will take
some time to build and ship their individualized tapes to all sites.
They will be pleased to move you to the top of the shipment list, if
you simply contact your SAS Salesperson and request early shipment.
Merrill Consultants STRONGLY RECOMMENDS IMMEDIATE INSTALLATION OF
SAS 6.07 as full replacement for SAS version 5. It's REALLY THAT
GOOD, and it's even better than the benchmarks in Newsletter TWENTY!
2. PROC CONTENTS DATA=PDB._ALL_ DETAILS; under SAS 6.07 now provides
the number of observations and number of variables, one per line,
similar to the SAS 5.18 contents. It is said that SAS 6.08 will
add the dataset size to the DETAILS option information.
3. SAS 6.07 has cleaned up their %MACRO compiler significantly. The
compile CPU time for ANALDB2R with and without input data:
0-obs 75-acct,34-stat
SAS 5.18 27.7 29.7
6.06 261.4 267.9
6.07 58.1 61.4
(Note that these are the SAS Session CPU times, and do include the
CPU time for the %MACRO compile. The individual Data and Proc step
CPU times DO NOT INCLUDE the compile time. In one ANALDB2R run with
the default reports plus Statistics Detail, and only a few hundred
records, the %MACRO compile time was 47% of the step total!)
4. If you run out of disk space in your WORK file while processing
%MACRO definitions, you may get SAS messages:
THE SAS SYSTEM IS UNABLE TO OPEN A MACRO SYMBOL TABLE. and/or
UNABLE TO WRITE HEADER INFORMATION FOR CATALOG WORK.SYSST3.
indicating you need to increase the WORK space allocation.
5. If you experience strange error messages under SAS 6.06/6.07,
especially with a BUILDPDB that has been tailored to process many
additional SMF records, increase the value of MEMSIZE (some sites
have raised it to 24MB, one site raised it to 32MB, and one of my
own stress test programs that compiles every SMF record on the face
of the earth required 56MB!). When SAS runs out of virtual memory
in its compile phase, it may produce unrelated messages concerning
a SAS VIEW, or Not Enough MFEs, etc., etc.
Added in 1999: The default MXG MEMSIZE has been 48MB for some time;
make sure your REGION=48MB is specified to let SAS get all 48MB.
Some very old MXG examples show REGION=4M which no longer works!
The Not Enough MFEs can happen with 6.08 and 6.09, too.
6. SAS 6.06 has been repaired, and can now be installed and used.
SAS 6.06 has been repaired, as long as you have installed the
October, 1991, or later, SAS Usage Notes tape maintenance, which
contains an IEBCOPY load library with ALL SAS ZAPs pre-applied for
all SAS products. The following resolved problems should be noted:
One site reported that they received an OC1 when reading an SMF file
but by removing Z6062909 (on the December SAS tape), the OC1 went
away. Unfortunately, Z6062909 was created to force the OC1 ABEND,
so that you would know you had had an I/O error when SAS discovered
that some I/O errors occurred that returned a zero condition code!
SAS is still working on a real fix to the I/O error problem.
SAS Z6063476 (on SAS October tape). Unable to read a SAS dataset
that was built by Version 5.18 using Tape format (i.e., built with a
DDNAME of TAPE..., as MXG's MONTHBLD does to eliminate tape mounts).
The job fails with "VARIABLES NOT FOUND" error, then 0C4 ABENDs.
The ZAP corrects the error, but a circumvention which allowed you to
read the dataset was to specify "LIBNAME TAPE V5SEQ;" and use the
DDNAME of TAPE to point to the disk dataset in tape format. As an
aside, specifying "LIBNAME TAPETEMP TAPE;" (used in MONTHBLD for
SAS 6.06 compatibility) causes the disk dataset in tape format to
be created with a BLKSIZE of 32760, which requires more disk space
than does half-track blocksize. Unfortunately, it does not appear
possible to change this blocksize, but this has only minor impact,
since TAPETEMP is only temporary SYSDA space.
There are still occasional 6.06 ABENDs in WORK dataset processing.
Three additional SAS ZAPs on their OCTOBER 1991 SAS USAGE NOTES tape
seem to have corrected most remaining problems. The critical zaps
are Z6063169, Z6063214, and Z6063409. One site experienced without
these zaps reported its strange ABENDs went away when BUFNO=2 was
changed to BUFNO=3 on their WORK DD!
7. The following important notes are repeated from prior Newsletters:
a. SAS OPTIONS REQUIRED under version 5.18, 6.06, or 6.07.
Please read this section carefully. MXG execution will fail if you
do not pay attention to these (potentially incompatible) changes:
SAS options that are MANDATORY with all SAS Versions:
NOIMPLMAC MAUTOSOURCE SASAUTOS=SOURCLIB ERRORABEND MACRO DQUOTE
NOIMPLMAC should ALWAYS be used in ANY program.
MAUTOSOURCE and SASAUTOS=SOURCLIB are required by MXG to
resolve %MACRO references without extra I/O by using autocall.
ERRORABEND ensures SAS stops on any error condition.
MACRO enables SAS to recognize %MACROs.
DQUOTE is needed to extract values of certain string %MACROs.
SAS options MANDATORY under SAS Version 5.18 (do not exist in V6):
MWORK=28000 GEN=0
MWORK was needed in large %MACROs (like %ANALDB2R).
GEN=0 was needed to eliminate unnecessary I/O and CPU time, and
is discussed in the 1984 Guide (see Index).
SAS options MANDATORY under SAS Version 6 (did not exist in 5.18):
MEMSIZE=24M
Previously, MXG recommended MEMSIZE=12M, but programs that build
many datasets concurrently (like BUILDPDB, VMACVMXA, etc.) will
need more of this virtual storage above the line, because of the
new MXG recommended value for BLKSIZE='half-track'. The MEMSIZE
option only works under MVS/XA and MVS/ESA, and since it is not
a real resource, and only used when needed during the "building"
phase in MXG processing, the new default of MEMSIZE=24M should
not cause any problem, and will avoid unnecessary ABENDs.
If you are using MXG and SAS 6.06 under MVS/370 or CMS/370, you
will have to reduce this MEMSIZE= parameter to no more than 6M,
and must reduce the BLKSIZE (try 4096, or the minimum 1024).
Small BLKSIZE will allow your program to compile, but the DASD
work space you will need will significantly increase, as will
the run-time, IOs, and CPU requirements for the same job.
SAS options STRONGLY RECOMMENDED for SAS 6.06 performance:
BLKSIZE=23040 BUFNO=2 -- for 3380's
BLKSIZE=27648 BUFNO=2 -- for 3390's
Both are now automatically set by MXG's use of BLKSIZE(DASD)=HALF
in MXG's CONFIG/CONFIG07 members for permanent datasets, but
note that the WORK DD in the default SAS JCL procedure contains
a BLKSIZE=6144 parameter which MUST be overridden or changed for
optimum execution performance:
// EXEC SAS
//WORK DD DCB=BLKSIZE=23040
The BLKSIZE option in the CONFIG member will have no effect if a
DCB=BLKSIZE value is specified on the JCL DD statement.
See "SAS Benchmarks" in Newsletter TWENTY for resource savings.
SAS options recommended with either SAS Version 5.18, 6.06 or 6.07:
FIRSTOBS=1 OBS=MAX
ERRORS=2
NOSOURCE NOSOURCE2 NOMACROGEN NOMPRINT NOMLOGIC
So how do you specify these recommended/required MXG options?
In Version 5.18, MACRO and MWORK=28000 must be specified on the EXEC
statement, but all other options could be specified on an OPTIONS
statement right after your SYSIN DD * statement. However, so you do
not have to remember them nor type them, MXG member SASOPTV5 has all
of the recommended options in an OPTIONS statement, so you need only
%INCLUDE SOURCLIB(SASOPTV5); as your first SAS statement:
//stepname EXEC SAS518,OPTIONS='MACRO MWORK=28000'
//SYSIN DD *
%INCLUDE SOURCLIB(SASOPTV5);
... the rest of your program ...
For SAS Version 6, options can be set with an OPTIONS statement, or
with the OPTIONS= JCL parameter, or (as is MXG's recommendation) via
the CONFIG= JCL parameter on the EXEC statement. MXG member CONFIG
contains the MXG required and recommended options for SAS 6.06, and
member CONFIG07 contains the SAS 6.07 recommendations. The BLKSIZE
and MEMSIZE options were increased in MXG 9.9, and the new CONFIG07
member was added with new SAS 6.07 options. (You could also supply
options by overriding the CONFIG DD in the SAS JCL procedure, but
using the CONFIG= parameter is safer and simpler.).
// EXEC SAS606,TIME=10,
// CONFIG='MXG.SOURCLIB(CONFIG)'
// EXEC SAS607,TIME=10,
// CONFIG='MXG.SOURCLIB(CONFIG07)'
IF YOU DON'T HAVE THE RIGHT OPTIONS IN EFFECT, YOU WILL RECEIVE
RUDE AND INSULTING SAS ERROR MESSAGES, INCLUDING 180 ERRORssss!
b. Format libraries are different in MVS SAS Version 6 and Version 5.
The MXG-built "SASLIB" formats are built by the first step of either
JCLTEST (for SAS 5.18) or JCLTEST6 (for SAS 6.06).
Under SAS Version 5.18, formats are members of a PDS load library
which must be allocated as SPACE=(CYL,(3,1,99)). SAS 5.18 formats
are always referenced using the DDNAME of SASLIB.
Under SAS Version 6 formats are members of a SAS data library, which
must be allocated as SPACE=(CYL,(1,1)). Note there is NO third SPACE
parameter (for PDS directory blocks), because data libraries in SAS
Version 6 are physical sequential files. SAS Version 6 format
libraries are always referenced using the DDNAME of LIBRARY.
In either version of SAS, the blocksize is set by the PROC FORMAT.
MXG always requires the appropriate DDNAME (SASLIB or LIBRARY).
You will fail with SAS 170 FORMAT NOT FOUND errors if you do not
have the correct format library pointed to by the correct DDNAME.
VI. Installation Instructions for MXG Version 9.9.
Over 800 sites have installed a PreRelease of Version 9, with almost
no reported problems. Sites migrating from MXG Version 8.8 or later
have found installation of MXG 9.9 was smooth and easy. The only
known incompatibilities are listed below, before the instructions.
Under ALL SAS Versions:
BUILDPDB/3 sites who have added DB2 processing to their PDB must now
"back-out" their exit code as described in Change 9.175; MXG now has
added DB2ACCT/DB2STAT0-1 datasets automatically to your PDB, and
a syntax error in BUILDPDB will occur if you fail to heed this note.
Only under SAS Version 6.07:
Use new member CONFIG07 instead of CONFIG. No error will occur, but
several unnecessary WARNING messages will be eliminated by use of
the new CONFIG07 member for 6.07.
Only under SAS Version 5.18:
BUILDPDB/3 under SAS 5.18 ONLY (not required with SAS 6.06/SAS 6.07)
sites must add a new temporary DD card in their JCL for BUILDPDB:
//SMFTEMP DD UNIT=SYSDA,SPACE=(CYL,(20,20))
This inconvenience may help eliminate SAS 5.18 compiler 344 errors.
If you encounter SAS 5.18 errors 344 or 160 see Change 9.175.
A syntax error in BUILDPDB will occur if you fail to heed this note.
However, if you have NOT installed MXG 8.8, you MUST note:
a. See SAS Notes section of this newsletter. The SAS options that
are required by MXG were changed between MXG 7.7 and MXG 8.8.
b. Execution under SAS 6.06 is different than under SAS 5.18. See
SAS Notes section of newsletters 17-21.
e. To write your PDB directly to tape, IMACCICS must be changed as
described in comments in the member. Although MXG does support
creation of the PDB directly to tape, most sites find it better
(especially elapsed time) to create the PDB on temporary disk and
then use PROC COPY to copy from disk to tape. (BUILDPDB re-reads
PDB datasets to build RMFINTRV, and this can cause lots of tape
spinning when the PDB DDname points directly to tape.)
Always read comments in the CHANGES member for compatibility issues, as
well as for any last minute changes added after Newsletter composition.
1. Installation, re-installation, and Space Requirements.
The MXG Installation instructions are found in Chapter 32 of the MXG
Supplement for both new installation and for version replacement, and
those instructions, as modified herein, are still valid.
The MXG tape is distributed as a Non-Labelled (NL) tape containing a
single file with DCB=RECFM=FB,LRECL=80,BLKSIZE=32720. The MXG tape is
an unloaded Partitioned Dataset in IEBUPDTE format. Note that the tape
blocksize was changed to 32720 (it had been 6160 in prior versions). You
will receive I/O errors if you specify the incorrect blocksize.
Under MVS use IEBUPDTE utility to build the MXG.SOURCLIB PDS library.
Under CMS use TAPPDS command to build the SOURCLIB MACLIB library.
Allocate the MXG.SOURCLIB for MXG 9.9 with SPACE=(CYL,(50,1,299)), using
DCB=(RECFM=FB,LRECL=80,BLKSIZE=23440) on 3380 devices, or
DCB=(RECFM=FB,LRECL=80,BLKSIZE=27600) on 3390 devices.
Under CMS, approximately the same space (50 cylinders) will ultimately
be required by the MXG SOURCLIB MACLIB, but during the CMS installation
process you will need at least 100 free cylinders on your minidisk.
MXG is tailored and extended by "Installation Macro" members (members
beginning with IMAC....), and by "MXG Exit Facility" members (members
beginning with EX....) that are copied and edited into the installation
"USERID.SOURCLIB", the name of the "MXG Tailoring" library, that you
create and concatenate ahead of MXG.SOURCLIB to the SOURCLIB DDNAME:
//SOURCLIB DD DSN=USERID.SOURCLIB,DISP=SHR --> site tailoring (yours)
// DD DSN=MXG.SOURCLIB,DISP=SHR --> never changed (mine)
If this is an MXG re-install, there should already be a USERID.SOURCLIB.
If not, then allocate one using the same attributes as the MXG.SOURCLIB,
with SPACE=(CYL,(1,1,99)). Under CMS create an equivalent MACLIB.
Tailor by editing members that you copy from my library to your library.
IMAC.... members are self-documenting, and member IMACAAAA indexes all
IMAC members. Most MXG sites have only a few tailoring members in their
USERID.SOURCLIB (typically, IMACACCT, IMACSHFT, IMACWORK).
VMAC.... members contain the actual processing code, and normally should
not exist in your USERID.SOURCLIB, and should always be removed when you
install a new version of MXG. However, if you have installed printed
changes from an MXG Newsletter, you might have copied VMAC member(s)
from MXG.SOURCLIB into your site's USERID.SOURCLIB and then modified the
VMAC member. (Alternatively, you could have created an MXG.CHANGLIB in
which you put those in-between-version changes.) In either case, if
you made temporary changes, they must be removed now. Delete all of the
VMACs members from your USERID.SOURCLIB (or delete the MXG.CHANGLIB).
Otherwise, your modified VMAC member would be used instead of MXG's.
You should record all tailoring changes in your USERID.SOURCLIB so the
next MXG technician will know what you have done. You should create a
member named CHANGES in your USERID.SOURCLIB, similar to member CHANGES
in the MXG.SOURCLIB which documents all MXG changes.
If there are IMAC.... members in your USERID.SOURCLIB, you must look at
the alphabetic list of changed IMACs in the Change Log, below, and must
retrofit your tailoring on the new member in this library. In MXG 9.9,
only IMACACCT was changed (Change 8.290, in member CHANGES).
Whenever you install changes or test a new version of MXG (or even your
own reports), be extra careful to look on the SAS log for any real error
conditions. Search for all occurrences of "ERROR:" and "ERROR :" and
"UNINITIALIZED" and "NOT CATLGD", as they may indicate a serious error.
A PROC PRINT and a PROC MEANS of each new MXG-built SAS dataset can help
you to understand their contents, and should be used to examine any
unusually large, negative, or suspicious values. Print all variables in
the dataset, and read the variable's descriptions in Chapter FORTY.
Summary of critical actions to be taken in installing new version:
a. All VMAC.... members in your USERID.SOURCLIB must be examined
and, in general, must be deleted.
b. All IMAC.... members in your USERID.SOURCLIB must be compared
with the IMAC.... members that have been changed in this MXG
version (see alphabetic list at beginning of Change Log). You
you MUST start with the IMAC in this version and retrofit your
installation's tailoring. Member IMACAAAA indexes all IMAC's.
c. It is always wisest to PROC PRINT the first 50 observations of
important datasets, especially PDB.JOBS, which can be affected
by user tailoring in IMACPDB. A visual scan of that PROC PRINT
serves as an excellent validation of correct installation, and
will almost always detect any serious problems BEFORE you begin
your production MXG runs! See also the MXG utility UTILPRAL.
VII. Documentation of MXG Software.
Member CHANGES always contains the version number of MXG Software, and
it lists changes that were installed in that version. The members named
CHANGEnn have the contents of CHANGES when that "nn" MXG version was
created. Description of enhancements will be found in the text of the
Change description that made the enhancement (in those CHANGES and
CHANGEnn members). The CHANGE members can be scanned online (with SPF
BROWSE) to search for specific product name references (CICS, MVS/ESA,
etc.). The text of each Change identifies the member(s) that were added
or altered by that change. Documentation (especially for new product's
support) is usually also found in comments at the beginning of those
members listed in the change entry.
Member NEWSLTRS contains the text of all newsletters (up through the
newsletter that accompanied that MXG release). You can search NEWSLTRS
for product name or acronym to find the technical notes, APARs, etc.
from all MXG newsletters. (The Change Log of each Newsletter is not
replicated in member NEWSLTRS, since that text will be in CHANGES).
Member DOCVERnn is the "delta-documentation" (in abbreviated Chapter
FORTY style) of only those variables and datasets that were changed
between successive MXG Versions. There is a DOCVERnn "delta" member in
the MXG library for each version.
Penultimately, member DOCVER contains abbreviated Chapter FORTY format
that documents all of the variables names in all of the MXG datasets
that are created by that MXG Software Version (alphabetically by data
set name and variable name).
Finally, MXG is a source distributed system, so you can often find your
answer by BROWSE/EDIT of the source member, especially the VMACs that
actually create the dataset, or ANALs that analyze the MXG datasets.
In many instance, the MXG Variable name is the IBM or Vendor's field or
DSECT field name. In other cases, the DSECT field name is carried as a
comment beside in the MXG INPUT statement to map field name to MXG's
variable name. MXG expects you to also have access to the vendor's
documentation of particular data records you are using for analysis.
VIII. Change Log
--------------------------Changes Log---------------------------------
You MUST read each Change description to determine if a Change will
impact your site. All changes have been made in this MXG Library.
Member CHANGES of the MXG SOURCLIB will always be more accurate than
the printed changes in a Newsletter, because the software is created
after the newsletter is sent to the printer!
Member CHANGES always identifies the actual version and release of
MXG Software that is contained in that library.
The actual code implementation of some changes in MXG SOURCLIB may be
different that described in the change text (which might have printed
only the easily installed, critical part of the correction).
Scan each source member named in any impacting change for any comments
at the beginning of the member for additional documentation, since the
documentation of new datasets, variables, validation status, and notes,
are often found in comments in the source members.
Changes thru 9.229 are contained in MXG Version 9.9 Mar 1, 1992.
Changes thru 9.218 were contained in MXG PreRelease 9.8 Feb 3, 1992.
Changes thru 9.212 were contained in MXG PreRelease 9.7 Jan 27, 1992.
Changes thru 9.205 were contained in MXG PreRelease 9.6 Jan 14, 1992.
Changes thru 9.184 were contained in MXG PreRelease 9.5 Dec 1, 1991.
Changes thru 9.175 were contained in MXG PreRelease 9.4 Nov 15, 1991.
Changes thru 9.107 were contained in MXG PreRelease 9.3, Aug 1, 1991.
Changes thru 9.104 were printed in NEWSLETTER TWENTY Aug 1, 1991.
Changes thru 9.069 were contained in MXG PreRelease 9.2, July 1, 1991.
Changes thru 9.038 were contained in MXG PreRelease 9.1, June 3, 1991.
Changes thru 8.305 were contained in MXG Production 8.9, May 1, 1991.
Changes thru 8.285 were contained in MXG Production 8.8, Apr 8, 1991.
Changes thru 8.283 were printed in NEWSLETTER NINETEEN, Apr 8, 1991.
Alphabetic INDEX of significant changes in MXG 9.9 (after MXG 8.8):
Member Change Description
many 9.151 DASD Device 3345 ("Serpentine") data recognized.
many 9.152 Tape Device 3490E ("Serpentine") data recognized.
ANALACHE 8.293 Cache RMF Reporter analysis uses 3990 records now.
ANALDBTR 9.135 DATASET S064058 NOT SORTED error corrected.
ANALDB2R 8.299 Variable Not Found if only Acct Trace Long requested.
ANALDB2R 9.030 DB2 Reports have incorrect values for some fields.
ANALDB2R 9.032 DB2 Audit Reports not created if PDB=SMF specified.
ANALDB2R 9.104 DB2 Trace TRANSIT report causes DATA IS NOT SORTED.
ANALDB2R 9.210 DB2PM-like SQL trace reports added.
ANALRACF 9.064 RACF Analysis of OPERATOR,SPECIAL activity.
ANALTMS 9.059 PROC SUMMARY out of memory condition.
ANALVVDS 9.031 PERM should be changed to MXGVVDS.
ANALVVDS 9.053 MXG 9.1 only, VMXGVVDS should have been MXGVVDS.
APAR/PTF 9.141 OY33312/UY52529 has no impact on MXG processing
ASUMDBDS 9.012 TMON/CICS trending fails with Release 7 data.
ASUMJOBS 8.291 EXCPTOTL, IOTMTOTL were not kept in BATCHJOB.
ASUM70PR 9.091 TYPE70PR data summarized into PDB.ASUM70PR
ASUM70PR 9.179 ASUM70PR data wrong if NRPRCS not equal to PARTNCPU.
AS400PDS 8.286 AS400PDS contains no records.
BLKSIZE 9.109 MXG distribution tape block size changed to 32720.
BUILDPDB 9.049 SAS 6.06 and MXG 8.8 under MVS/370 options.
BUILDPDB 9.095 Dataset TYPE0203 added to default datasets built
BUILDPDB 9.175 DB2ACCT,STAT0/1 automatically created by BUILDPDB.
CASORT66 8.295 SAS datasets destroyed by CASORT release 6.6.
CHANGE08 9.073 Example to create _DAY in 8.213 was wrong
CONFIG 9.022 Option VERSIONLONG specified to display SAS version.
CONFIG 9.131 Default CONFIG for 6.06 updated.
CONFIG07 9.131 Default CONFIG for 6.07 updated.
DIFFDB2 9.080 Not all DB2STAT0/1 variables were deaccumulated
DIFFDB2 9.128 DB2STAT0/1 variables improperly deaccumulated.
EXPDB304 9.034 BUILDPDB/3 steps data creation exit.
EXPDB6 9.034 BUILDPDB/3 steps data creation exit.
EXTY72 9.184 CPURCTTM invalid in TYPE72, not included in CPUTM.
IEBUPDTE 8.286 Unload of MXG 8.8 set condition code 4.
IMACACCT 8.290 ACCOUNT FIELD n LENGTH WRONG corrected.
IMACCICS 9.005 PDB cannot be built on tape unless IMACCICS changed.
IMACDB2 9.121 DB2 Correlation ID decoding corrected.
IMACEXCL 9.051 Support for CICS type 110 EXCLUDE statement.
IMACEXCL 9.145 CICS EXCLUDE/INCLUDE logic externalized
IMACFMTS 9.066 New IMAC for user formats for SAS 6.06.
IMACICDA 9.146 CICS EMP data control externalized
IMACICDB 9.181 Support for APAR PL83370 in DBCTL data with CICS/ESA
IMACICOx 9.146 Omegamon V550 User EMPs added to type 110
IMACKEEP 9.017 A better way to drop MXG variables not using IMACKEEP
JCLDASD 9.031 MXGDASD,PERM DDNAMEs should be DISK,MXGDASD
JCLMNTH 9.225 MONTHBLD require only one tape drive, runs faster!
JCLTEST6 9.108 FORMAT not found error in step SMFSMALL.
READDB2 9.164 List processing %MACRO for DB2 IFCIDs added.
RMFINTRV 9.184 Enhanced, MVS/ESA CPU times and PR/SM data added.
RMFINTRV 9.204 RMF72D NOT SORTED condition corrected.
SPUNJOBS 9.005 PDB.SPUNJOBS on tape caused 207 error.
SPUNJOBS 9.013 SPUNJOBS timestamps not 8 bytes, truncating values.
TLMS 9.041 TLMS causes 713-04 ABEND if DBLTIME=0.
TRNDDB2A 8.301 DB2 Acct trending failed in 2nd week of execution.
TRNDDB2A 9.209 DB2 Trending errors corrected.
TRNDDB2S 8.301 DB2 Stats trending failed in 2nd week of execution.
TRNDVMXA 9.028 VM/XA Trend Data Base implemented.
TRNDVMXA 9.089 VM/XA Trended PFXUTIME and PCTCPUBY incorrectly
TRND70PR 9.091 TYPE70PR data creates new TREND.TRND70PR
TRND70PR 9.179 TRND70PR data wrong if NRPRCS not equal to PARTNCPU.
TRND71 9.092 TYPE71 data creates new TREND.TRND71
TRND72 9.093 MVS/ESA 4.2 variables added to TREND.TRND72
TYPEAPAF 9.218 Support for Amdahl's APAF.
TYPECIMS 9.122 IMF Chained Transactions response time corrected.
TYPECIMS 9.235 Support for IMF 1.7 log records.
TYPEIMS 9.106 IMS Multi-trans resources wrong.
TYPEIMS 9.182 Multi-trans corrections, CONDCODE no longer blank.
TYPEIMS 9.216 IMS 3.1 resources wrong for Quick Reschedule trans.
TYPEMON8 9.011 TMON/CICS Release 9.0 supported via conversion only.
TYPEMON8 9.015 TMON/CICS Version 8 History file cause MXG failure.
TYPEMON8 9.168 STOPOVER with bad record in Landmarks Monitor
TYPEPDL 8.297 Fujitsu FACOM MSP and FSP support replaced XPDLPDA.
TYPE84 9.202 JES3 JMF type 84 SMF record partial support.
UTILCICS 9.019 Not Sorted error corrected for CICS/ESA dictionary.
VDOCACHE 9.027 Documentation re-write has begun.
VMACACF2 8.289 ACF 5.2 new variables not kept.
VMACACF2 9.086 ACF2 REC=L CN=3 records were not output
VMACACHE 8.293 Cache RMF Reporter support enhanced and decoded.
VMACAIM2 8.298 Fujitsu AIM database manager records corrected.
VMACCADM 9.021 Support for CADAM's Statistics Data File.
VMACCCC 9.172 Softtool Corporation's CCC (Change Control) user SMF
VMACCMA 9.143 Support for CMA-SPOOL user SMF record
VMACCMF 9.126 CMF User SMF record STOPOVER corrected.
VMACCTLD 9.038 Support for 4th Dimension Sofware's CONTROL-D record.
VMACDB2 9.138 Support for DB2 2.3.0
VMACDCOL 8.285 IBM DASD measures use MB for million, not megabytes.
VMACDCOL 9.144 DCOLLECT values incorrect after IBM APAR OY36364
VMACDCOL 9.157 DCOLLECT variables changed from character to numeric
VMACDCOL 9.180 Capacity values off by 120 bytes.
VMACDMON 9.196 Support for ASTEC 1.5.
VMACEPIL 8.284 Support for EPILOG/CICS 451 added, and enhanced.
VMACEPIL 8.287 Invalid member on MXG 8.8 replaced.
VMACFOCU 9.223 Support for Information Builder's FOCUS MSO SMF.
VMACFTP 9.056 Support for NetView FTP User SMF record.
VMACFTP 9.170 FTP records produce no observations
VMACHSM 9.097 Support for HSM 2.6 SMF record
VMACILKA 9.020 Support for Interlink's SNS/TCPaccess SMF record.
VMACILKG 9.020 Support for Interlink's SNS/SNA Gateway SMF record.
VMACILKV 9.020 Support for Interlink's SNS/TCPvt SMF record.
VMACIMS 9.063 TYPEIMS Input Queue time correction.
VMACLMS 9.229 Support for Memorex Telex LMS/PM SMF record.
VMACMIM 9.173 LEGENT's Multi-Image Manager (MIM) user SMF record
VMACMIM 9.173 Support for LEGENT's Multi-Image Manager MIM record.
VMACNETP 9.116 NPM 1.4.1 support was not complete in MXG 9.3.
VMACNETP 9.118 Support for NET-PASS user SMF record.
VMACNSM 9.194 Support for NOMAD's Session Manager record.
VMACNSPY 9.033 STOPOVER protection for invalid records.
VMACNSPY 9.044 STOPOVER with NETSPY Accounting record.
VMACNSPY 9.045 NETSPY Accounting subtype caused STOPOVER.
VMACNSPY 9.165 LEGENT's LANSPY component of NETSPY additions.
VMACOMCI 9.147 Omegamon V550 User SMF record
VMACOPCA 9.224 IBM's OPC/A Job Tracking and Audit Trail Log.
VMACRMDS 9.139 RMDS records RMDSORG='D' STOPOVER corrected.
VMACSMF 9.094 New message at end of SMF processing on log
VMACSNAP 9.167 Codework Software's SNAPAD-MVS user SMF record
VMACSPMS 9.088 Support for Amdahl's SPMS Release 1.2 SMF record
VMACTAO 9.018 Support for Fischer's Totally Automated Office TAO.
VMACTAO 9.200 Support for Emc2/TAO Release 3.3.0.
VMACTMS5 9.016 Support for CA-1 (TMS) Release 5 complete rewrite.
VMACTMS5 9.057 Missing semicolons.
VMACTMVS 9.142 TMON/MVS spanned record STOPOVER circumvented
VMACUCB 9.002 3490E cartridge tape now recognized.
VMACVMXA 8.296 VM/XA used more work space than really required.
VMACVMXA 9.096 LOGICAL RECORD SPANS message corrected
VMAC102 9.178 IBM incompatible change to IFCID 140, 141 in 2.3
VMAC110 8.292 Unexpected Statistics Subtype due to Boole CICSMGR.
VMAC110 9.051 Support for Shared Medical Systems CICS modifications
VMAC110 9.062 Support for CICS/ESA 3.2.1.
VMAC110 9.084 CICS PCLOADCN has different meaning.
VMAC23 9.134 New fields were not input.
VMAC28 9.061 Support for NetView Performance Monitor NPM 1.4.1.
VMAC28 9.214 Support for NPM Release 1.5.
VMAC30 9.001 INVALID DATA FOR SMF30ASO with MVS/ESA 4.2 eliminated
VMAC30 9.085 STCs starting with A confused APPC logic.
VMAC30 9.134 Support for MVS/ESA 4.2.2.
VMAC42 9.048 Circumvention of IBM DFP 3990 Cache type 42 errors.
VMAC57 9.010 Invalid ESS Section message due to IBM error.
VMAC6 9.083 OUTDEVCE changes affect PRINTLNE, EXTWTRLN
VMAC6 9.154 LEGENT's Bundle product caused type 6 stopover
VMAC6156 9.046 TYPE6156 variables ACTION, FUNCTION not valid.
VMAC7072 9.001 INVALID DATA FOR IOCRFC with MVS/ESA 4.2 eliminated.
VMAC7072 9.054 MXG 9.1 only, Change 9.001 incompletely installed.
VMAC7072 9.070 TYPE72 CPUHPTTM,CPUIIPTM,CPURCTTM wrong in MVS 4.2
VMAC7072 9.072 TYPE70PR LCPUADDRs 2 and 3 wrong if DEDICATED CPU
VMAC7072 9.119 ACF2 STOPOVER due to bad record length corrected.
VMAC7072 9.120 ESA CPU times HPT/IIP/RCT wrong by 2%.
VMAC73 9.001 INVALID DATA FOR IODFDATE in MVS/ESA 4.2 eliminated.
VMAC74 9.001 First STOPOVER with MVS/ESA 4.2 data corrected.
VMAC74 9.039 Second STOPOVER with MVS/ESA 4.2 data corrected.
VMAC78 9.055 STOPOVER with MVS/ESA 4.2 data corrected.
VMAC78CU 9.047 Missing fields due to IBM microcode errors.
VMAC79 9.007 Support for (archaic?) RMF 3.5.1 type 79 records.
VMAC90 9.134 Support for MVS/ESA 4.2.2 added new subtypes.
VMXGHSM 9.009 HSM MCD data can cause STOPOVER.
VMXGVPD 9.158 STOPOVER with VPD record type "E".
VMXGVTOC 9.159 VTOC data beyond 3rd extent lost since MXG 7.2.
VMXGVTOF 9.035 SAS 5.18 only, did not read all extents.
VMXGVTOF 9.040 First Extent on each volume lost. .
VMXGVTOF 9.159 VTOC data beyond 3rd extent lost since MXG 7.2.
WEEKBLD 9.050 Submitting job may need DCB on INTRDR DDNAME.
XMAC74 9.054 MXG 9.1 only, Change 9.001 incompletely installed
Inverse chronological list of all Changes:
Note that the newsletter went to the printer on Feb 14. There may
well be additional changes before the tapes are built the weekend
of February 23. Please read member CHANGES of the MXG 9.9 library
to see if any additional enhancements or changes were made while
the newsletter was at the printer.
Changes 9.235-9.105 were printed here in Newsletter TWENTY-ONE
See member CHANGE09 for actual change text.
End of Changes Log in Newsletter TWENTY-ONE