****************NEWSLETTER FIFTEEN**************************************
MXG NEWSLETTER NUMBER FIFTEEN November 11, 1989
Technical Newsletter for Users of MXG : Merrill's Expanded Guide to CPE
TABLE OF CONTENTS
I. MXG SOFTWARE VERSION 7 ANNOUNCEMENTS pages 2 thru 7
1. Prerelease MXG 7.2 now available upon request. page 2
2. Production MXG VERSION 7 planned for early 1990. page 2
3. Summary of new products and enhancements in MXG Version 7. page 2
4. Summary of critical changes described in Change Log. page 3
5. Enhancements still under development for MXG Version 7. page 4
II. TECHNICAL NOTES pages 7 thru 9
1. MVS/ESA changes to SMF Accounting AND RMF capture ratio. page 7
2. MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND. page 9
3. MVS SMF Type 26 Time stamps after JES2 Spool Transfer. page 9
4. MVS execution under VM, VM/XA, or PR/SM publication. page 9
5. MVS SMF Type 30 interval records missing for early ASIDs. page 9
6. MVS SMF Type 26 record corrupted. page 9
III. CHANGE LOG pages 10 thru 44
Changes through Change 7.165 to 7.036
IV. "The History of SMF" pages 45 thru 76
Paper presented at the 20th anniversary
of the Availability of the IBM SMF Product
at SHARE 73, Orlando, August, 1989.
COPYRIGHT (C) 1989 BY MERRILL CONSULTANTS DALLAS TEXAS
MXG IS A REGISTERED TRADEMARK OF MERRILL CONSULTANTS
SAS IS A REGISTERED TRADEMARK OF SAS INSTITUTE
I. MXG SOFTWARE VERSION 7 ANNOUNCEMENTS
1. Prerelease MXG 7.2 now available upon request.
The Prerelease of MXG Version 7.2 is now available upon request from
Merrill Consultants (or through your "MXG Representative" outside USA).
Request by phone (214-351-1966) or fax (214-350-3694) if you would like
the prerelease sent to you (at no cost). MXG is sent on 3480 cartridge
tape media, unless you specifically requested MXG on a 3420 tape reel.
The Beta MXG 7.0 was installed at 130 sites, and their feedback was
incorporated in the Beta MXG 7.1 sent to a small number of additional
sites. The MXG 7.2 is called a Prerelease now to indicate that it has
been validated sufficiently to replace MXG 6.6 at sites that need its
new product support, its enhancements or its corrections.
(Prerelease MXG 7.2 was created 10/19/89 thru Change 7.161).
(Beta test of MXG 7.1 was created 9/14/89 thru Change 7.140).
(Beta test of MXG 7.0 was created 5/31/89 thru Change 7.098).
2. Production MXG Version 7 planned for early 1990.
We still intend to ship the Production Version of MXG 7 in early 1990,
probably in early February. Unlike the prerelease, which is sent only
if you request it, the "Production Version" of MXG is automatically
sent to all supported sites, accompanied by an MXG Newsletter.
See below for the enhancements that are not yet available in 7.2, but
that should be available in the Production Version 7 when shipped.
3. Summary of new products and enhancements in MXG Version 7.2.
The following major enhancements are available in MXG 7.2. Details
are in the actual Change Description (below) and documentation is
also found in comments in the members that are cited in the change
and in DOCVER and DOCVERnn members of the actual MXG 7.2 source.
New Products supported:
1. MXGTMNT, the long awaited MVS/XA or MVS/ESA MXG Tape Mount Monitor.
2. 3490 and 9348 tape drives (cart and reel respectively) recognized.
3. ARBITER SMF records from Tangram product.
4. NETVIEW Release 2 Hardware/Session Monitor External Log Record.
5. 3480/3490 Tape Compression (IDRC) recognized.
6. JES2 mods to capture SYSOUT release timestamp in type 6 SMF record.
7. FILEAID SMF record from COMPUWARE product.
8. OMEGAMON Command Audit SMF record from Candle.
9. AION Development System SMF record from AION.
10. MDFTRACK SMF record from Amdahl MDF environment.
11. TLMS (Tape Library Management System) catalog records.
12. TMS (Tape Management System) catalog records.
13. RMF Monitor III VSAM data set records.
14. Numerous additional DB2PM-like reporting in ANALDB2R.
15. DB2 Audit Class trace type 102 records.
Enhancements to previously supported products:
1. Identification of IBM RMF from Boole and Babbage RMF records.
2. DB2 SQL text of user inquires captured.
3. ACF2 'ARE' data sets captured.
4. Additonal trending summaries and report examples.
5. CPU Serials added to RMFINTRV.
6. ROSCOE 5.6 support for variable number of complexity levels.
7. ANALDSET enhancement to include VSAM type 64 records.
8. VMXGVTOC enhanced to expect up to 128 extents (for VSAM).
9. NETVIEW Release 2 Hardware Monitor External Log SMF record.
10. NETVIEW Release 2 Session Monitor External Log SMF record.
11. VM/XA new fields introduced by PTF VM35971.
12. IMS 2.0 and IMS 2.1 response measurement corrections.
13. Step accounting fields added to PDB.STEPS.
14. New MVS/ESA CPU timings in step records.
15. Support for DOS/VSE Power 3.1.2.
16. Support for RACF 1.8
17. Validation and cleanup of the Trend Data Base and related macros.
18. Non-support of IMS 1.3 transaction processing. See Change 7.075.
19. Validation and cleanup of all reported MXG 6.6 errors.
4. Summary of critical changes described in the Change Log (below).
Critical changes (i.e., those that correct error conditions) include
these:
Change Member Description
7.142 VMAC7072 TYPE70 CPU under PR/SM may be wrong.
7.139 VMACSYNC SYNCSORT record formats decoded wrong.
7.123 VMACDB2 DB2STAT0 wrong after PL29461.
7.115 VMACACF2 ACF2 datasets contain zero observations.
7.108 TYPEMONI MONITASK "OVHD" transaction record short.
7.105 VMAC6 PSF FORM variable wrong.
7.103 VMXG.... MWORK=28K option required for %MACROs.
7.098 BUILDPDB PROTECT= option considerations.
7.083 RMFINTRV RMFINTRV to count DASD I/O only.
7.082 VMAC110 CICSTRAN MAXTASK/SHRTSTOR.
7.081 VMAC7072 TYPE72MN trashed under MVS/ESA.
7.078 VMAC77 TYPE77 trashed under MVS/ESA.
7.076 TYPEMONI MONISYST trashed.
7.065 VMAC102 STOPOVER on type 102 for several subtypes
7.054 VMAC28 STOPOVER on type 28 for subtype 5C.
7.050 VMACVMON Concatenated VM/Monitor wrong.
7.047 ANALAUDT Audit report corrections.
7.044 VMACIDMS IDMS 10.2 cleanup of retained values.
7.039 VMAC22 STOPOVER on type 22 for MVS/370.
7.038 XMAC7xxx Circumvention for SAS Error 344.
7.036 BUILDPDB JPURTIME not found in some circumstances.
(The following changes were previously reported in MXG Newsletter
FOURTEEN and their text is printed therein):
7.035 WEEKBLD Implementation considerations.
7.023 TYPEIDMJ IDMS Journal correction.
7.022 CMS SAS CMS DMSSOP3633 Code 04 OPEN error.
7.009 VMACVMXA Concatenated VM/XA Monitor files
7.008 BUILDPDB JES2 Spool Transfer Purge Records
7.007 MONTHBLD Syntax error
7.006 BUILDPDB PRENTIME inadvertently in DROP list
" WEEKBLD PRENTIME not in BY list
7.005 VMAC102 DB2 Trace 102 STOPOVER on subtype 58 or 87
7.004 VMAC28 NPM type 28 "Unexpected subtype" message
7.003 JCLHSM HSM included nonexistent XMXGHSM member
7.002 VMACCIMS IMF CPUTM calculation incomplete
7.001 TRND...,etc. Trend Data Base finger faults
5. Enhancements still under development for MXG Version 7.
The following New 7.xxx (new products) and Change 7.xxx (corrections)
are anticipated to be completed in the Production Version 7 of MXG.
New 7.xxx MXG Version 7 will support SAS Version 6.06 (now in beta,
Unknown expected to be available in the first half of 1990). The
XXX xx, 1989 MXG Newsletter accompanying Version 7 will discuss.
New 7.xxx DB2 Version 2 Release 2 (DB2 2.2) is now available, but
VMACDB2 documentation has not yet arrived to permit detail look
VMAC102 at the IBM changes. It is known that new IFCIDs and new
XXX xx, 1989 fields in existing IFCIDs are created in DB2 2.2.
New 7.xxx Landmark's TMON/MVS development is not yet started, but
unnamed documentation and test data has been received from the
XXX xx, 1989 vendor.
New 7.xxx EPILOG/MVS product records will be supported in the
Unnamed Production Version.
XXX xx, 1989
New 7.xxx HSM SMF record is partially decoded (Change 7.096) but
VMACHSM not all segments have been decoded yet.
XXX xx, 1989
New 7.xxx RMF Type 79 records subtypes 1 and 2 processing is not
VMAC79 completed and will not execute successfully yet.
XXX xx, 1989
Thanks to Tom Skasa, GE Medical Systems.
Thanks to Sterling Green, GE Capital.
New 7.xxx In a JES3 environment, the MXGTMNT Tape Mount Monitor
Unnamed does not create a record for the mounts issued by JES3
XXX xx, 1989 Main Device Setup, MDS. Those mounts are recorded by JES3
in TYPE25 observations. Further research in TYPE25 shows
there is only one TYPE25 record per job, and it contains
the number of mounts actually issued by JES3, but only a
single mount duration, from MNTTIME to VERFTIME of the
final mount completion. This new member will combine the
TYPETMNT (MVS Issued tape mounts) with TYPE25 extracts to
create one observation for every actual tape mount. For
multiple mounts by JES3, the second and subsequent mount
observation will only count that a mount occurred at the
MNTTIME; the duration and end of mount timestamps will be
set missing (since you have no idea of how long it really
took). It also deletes TYPE25 observations with TAPEMNTS
zero (happens when UNIT=,,,DEFER is specified, no real
mount occurs, but a type 25 is written by JES3). This
program does not yet exits (its in telephone alpha!) and
this change will be updated when it exists.
Thanks to Mark Hutchinson, Atlantic States Bankcard.
Change 7.xxx %INCLUDE SOURCLIB(IMACACCT) logic needs to be added to
VMAC26J3 control JES3-only type 26 ACCOUNT1 variable's length.
XXX xx, 1989
Thanks to Dick Clarke, British Airways, Heathrow.
Change 7.xxx %INCLUDE SOURCLIB(IMACACCT) logic needs to be added to
VMACIDMS IDMS TASBFLDn accounting variables (to be renamed to
XXX xx, 1989 ACCOUNT1-ACCOUNTn, with lengths defined in IMACACCT, as
for all other MVS accounting fields.
Thanks to Dick Clarke, British Airways, Heathrow.
Change 7.xxx IBM incorrectly stores values in PWCOUNT (hex values are
TYPEVM F0F8 F0F9 F0C1 or EBCDIC 08 09 0A for actual data counts
XXX xx, 1989 of 8, 9, and 10!) MXG will circumvent IBMs error.
Chnage 7.xxx RMDS records have reported counts in errors for some of
TYPERMDS the page count variables, but have not completed the
XXX xx, 1989 investigation.
Change 7.xxx EPILOG CICS CL1000 Versions 430 and 440 support is still
TYPEEPIL incomplete (Change 7.094). Test data and documentation
XXACEPIL have been received from the vendor. One site has reported
XXX xx, 1989 that the offset needed to be set to -4 to process the
Version 430 data, but I have not yet even validated that
possibility.
Change 7.xxx VM/370 USER Class data can have erratic large negative
VMACVMON values because one of the two-byte IBM counters overflow
XXX xx, 1989 (typically DRPCANUS) which causes MXG to falsely think a
LOGON event occurred. (Because IBM does not flag LOGON
event in the USER data, MXG must attempt to infer that it
happened.) MXG will be revised to use only four-byte IBM
counters in its LOGON event detection (inference engine?)
Other products which may be added to MXG in the future that have been
requested by users include these candidates:
Likely to be in MXG Version 7:
MVS/ESA 3.1.3 (hopefully to be in MXG Version 7)
MXGMENU Selection Macro Variable definitions.
When product becomes generally available:
CICS 3.1
IMS 3.1
New Devices
After SAS Version 6.06 is generally available on the Mainframe:
MXGMENU Menu Architecture (needs SCL, Screen Control Language)
Not likely in MXG Version 7:
JES3 Jes Measurement Facility SMF type 84 record.
VM/XA VMPRF product reports may be replicated in MXG.
NETVIEW File Transfer Program Release 1.0 "User" SMF Record.
FACOM PDL record processing.
TPNS activity log.
IMS Fastpath
II. TECHNICAL NOTES
1. MVS/ESA CHANGES TO SMF ACCOUNTING AND RMF CAPTURE
a. CPU Time measurements have changed with MVS/ESA as this schematic
(aligned vertically, but not scaled horizontally) drawing shows
exactly what is captured where:
TYPE 70 (RMF-captured CPU measurement of real or logical CPUs):
Elapsed Interval (Duration multiplied by number of CPUs)
________________________________________________________________
______________________________________________________ ---------
CPU CPU
Executing Waiting
________ ________ _________ ________ ________ ________
CPU 1 CPU 2 CPU 3 CPU 4 CPU 5 CPU 6
Busy Busy Busy Busy Busy Busy
______________________________________________________
Total Hardware CPU Busy Time (from Type 70 "non-wait")
TYPE 72 (summing only the control performance group's data) contains:
_____ _____ ------------------------------------------
RMF RMF Uncaptured
TCB SRB RMF CPU Busy
CPU CPU pre-ESA 3.1.3
Busy
___________ _____________________________ ------------
CPUTM Reportedly to be Uncaptured
pre 3.1.3 captured in ESA 3.1.3 RMF CPU
with 3.1.3
TYPE 30 (if all work started and ended in the interval) contains:
_____ _____ _____ _____ _____ _____ _____ ------------
SMF SMF SMF SMF SMF SMF SMF Uncaptured
TCB SRB TCB SRB IIP RCT HPT SMF
CPU CPU CPI CPI CPU CPU CPU CPU
_________________________________________
CPUTM = Sum of all CPU times in TYPE30
Three new CPU time durations (in addition to the existing four)
have been added to the SMF type 30 record, which is the source of
job accounting (and PDB.JOBS, STEPS, SMFINTRV, TYPE30_V, etc.):
CPURCTTM - Region Control Task (Quiesce/Restore/Swap), caused
by this TSO user's swapping.
CPUIIPTM - I/O Interrupt Processing (Second Level Interrupt
Handler, SLIH)
and one brand new CPU measurement:
CPUHPTTM - Hiperspace processing CPU time. HPT CPU time is NOT in
captured in existing CPU TCB or SRB fields. When SORT,
for example, begins using Hiperspace processing, the
TCB CPU time will be significantly reduced, because
function was moved from TCB to the new HPT facility.
DFSORT benchmarks showed examples of TCB reduced from
100 seconds, but that reduction required 25 seconds of
HPT time, so the true reduction was from 100 to 75. If
your billing system is using the CPUTM variable in MXG
(which has always contained the absolute total of ALL
CPU times found in the type 30 record, even though the
original 1984 book said it was only CPUTCBTM+CPUSRBTM)
you were wise and will have no immediate CPU billing
problems, but if you charge only for CPUTCBTM, you are
unwise and extremely vulernable to incorrect billing.
Of great concern to capacity planning in the preceding schematic
is the absence of these three new CPU measures in the Performance
Group measures in TYPE72 data. These CPU measures will be in the
RMF data for the newly announced MVS/ESA 3.1.3.
b. Additional MVS/ESA measurement data (already in MXG 6.6) included
TYPE 30: Step Hiperspace/Data Space Activity
HIPAGINS - Hiper Page Ins
HIPAGOUT - Hiper Page Outs
DSSIZHWM - Data Space High Water Mark, Megabytes
CREADMIS - CREADS Missed in Hiperspace
DIVRREAD - Address Space total Reread Count
TYPE 72: Performance Group paging/memory measures
PGPAGEIN - Page Ins (from Auxiliary DASD)
ACTFRMTM - Frame-seconds of memory occupancy
while task in this performance group
period were "SRM Resident"; includes
both Central Store and Expanded Store.
HIPPGINS - Hiperspace Page Ins (from Aux DASD)
HIPRDMIS - Hiperspace Read Misses (CREADS in 30s)
c. MVS/ESA also creates an RMF record from RMF Monitor III screens,
in a new subtype of Type 72 records, but only a small part of
the RMF III screen data is written to SMF. MXG's member JCLZRB
will process the screen data in the RMF III VSAM file, but the
TYPE72MN dataset is still valuable for a quick look, containing:
User Statistics Frame Statistics
AVGUSERS - Logged On MONACTV - Active Frames
AVGACTV - Active Users MONIDLE - Idle Frames
MONDUTR - Users Out and Ready MONDIV - DIV Frames
MONPAGE - Users Delayed Paging MONFIX - Fixed Frames
MONSWAP - Users Delayed Swap MONSLOT - Slots Used
MONPGIN - Page Ins
2. MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND
MVS SMF Type 30 Subtype 3 TCB/SRB times after 922 ABEND contain trash,
but because first bit is on, the number is seen by IBM as a negative
value and by MXG as a very large value (because MXG uses PIB4. instead
of IB4. for these fields). Now, IBM zeroes the TCB/SRB fields in the
record, with PTF UY24926. Thanks to Diane Epstein, SW Bell.
3. MVS SMF Type 26 Time stamps after JES2 Spool Transfer.
MVS SMF Type 26 Time stamps are wrong after reload of SYSOUT that was
previously offloaded with JES2 Spool Transfer program. APAR OY21221.
4. MVS execution under VM, VM/XA, or PR/SM publication.
The MVS under VM paper formerly available to your SE under PROFS that
was referenced in previous MXG newsletters, is now published in IBM's
"Operating MVS/XA in Multi-Image Environments", GG24-3274-01. This pub
has a number of additional topics. Must reading for PR/SM users as well
well as MVS under any form of VM!
5. MVS SMF Type 30 interval records missing for early address spaces.
Type 30 interval records are not created on tasks which are created
prior to the completion of SMF initialization. OZ96676 discusses this
design "feature". You may miss records on the catalog address space, as
it usually starts right after SMF, but before SMF has completed its own
initialization. There appears to be no fix at present, however the PLM
for Master Scheduler, LY28-1200-4 page 5-567 discusses why the SMF task
must WAIT before other address spaces! We've re-opened the problem with
IBM to see how to relocate the SMF and its WAIT ahead of Catalog ASID.
6. MVS SMF Type 26 record corrupted.
Type 26 JES2 records (ONLY for HJE3311 Rel C11) are invalid beginning
in the programmer name field, causing trashed variables in TYPE26. The
IBM fix is PTF OY21719.
III. CHANGE LOG
--------------------------Changes Log---------------------------------
You MUST read each Change description below to determine if a Change
will impact your installation. You can then either make the change
from the Change Description text, or you may wish to call Merrill at
214-351-1966 to request the prerelease of MXG 7.2, which already has
all of these changes installed and tested. Member CHANGES of the MXG
SOURCLIB will always be more accurate than the printed changes in a
Newsletter, and always identifies the MXG Version and creation date.
Printed changes may actually be implemented more comprehensively in the
actual MXG Software. Always read the comments at the beginning of each
source member named under the Change Number when you receive a new
Version of MXG. Documentation of new datasets and variables, validation
status, notes, etc., are usually in comments in the source members.
If you choose to install printed changes, you could copy the affected
member(s) from MXG.SOURCLIB into your existing USERID.SOURCLIB and then
make these changes therein. Alternatively, you can copy into a new PDS,
MXG.CHANGLIB, to hold these in-between-version changes, concatenating
MXG.CHANGLIB between USERID.SOURCLIB and MXG.SOURCLIB until you install
the next MXG replacement library. Then when you do install the next
version, you must then delete all of the members in the MXG.CHANGLIB
(if you used that technique), or you must individually remove only the
changed members from your USERID.SOURCLIB. It is always wisest to
document all of your changes in a member CHANGES in the USERID.SOURCLIB
or MXG.CHANGLIB library.
Whenever you install changes or test a new version of MXG (or even for
your own reports), be extra careful to look on the SAS log for any real
error conditions. Search for all occurrences of "ERROR:" "ERROR :"
"UNINITIALIZED" "NOT CATLGD", as they usually indicate a serious error.
PROC PRINT and PROC MEANS of each new MXG-built SAS dataset will go
a long way in explaining their contents, and can be examined to detect
unusually large, negative, or suspicious values.
If you are not having any problems reported herein (and many of these
changes only correct enhancements added in 7.0 or 7.1 or for leading
edge software), you can simply wait until the Production Version 7 is
automatically sent to your site next February.
Completed changes:
Change text is in member CHANGES or CHANGEnn, and not kept here to
save space. The text was actually printed in the physical newsletter.
--------Changes thru 7.165 were printed in NEWSLETTER FIFTEEN-----------
--------Changes thru 7.035 were printed in NEWSLETTER FOURTEEN----------
and are in member CHANGES of MXG Version 7 Library, which
becomes member CHANGE07 in the subsequent MXG version.
------------------------------------------------------------------------
------------------------------------------------------------------------
End of Changes Log, but how's this for page filler, printed verbatim
from an entry in IBM's INFO/SYS (SMF6CPS is COPIES in TYPE6):
E343725 (HIT LIST 000003/000013)
---------------------------------------------LINES: 1 THRU 15 OF
H E343725 S=TOOLS C=GY4 D=JUL89 E=JUL91 L=00015
TITLE: PSF NOT UPDATING SMF6CPS FIELD IN TYPE 6 SMF
RECORDS.
F -SUGG -OY21704--5665-27-501--IN-INCORR OU
REPORTED RELEASE R220
ERROR DESCRIPTION:
SMF6CPS IS NOT UPDATED CORRECTLY.
COMMENTS:
THIS APAR IS BEING CLOSED SUG (SUGGESTION). THE SUGGESTED
CHANGE WILL NOT BE IMPLEMENTED.
89/07/19,CHICAGO FS
$EOM
CME 20/20: The History of SMF
Session M710
August 21, 1989
H. W. Barry Merrill, PhD
Merrill Consultants
Dallas, Texas
SHARE Installation CM4
ABSTRACT
SMF became available 20 years ago with OS/360 Release 18. SHARE's 1964
SORC report was part of the input to IBM's 1966 SMF design document
(authored by an IBMer who had been a SHARE board member). The design
objective was two-fold: the stated SHARE requirements for control and
evaluation of the installation, and the unstated need for IBM to
understand how its OS and its program products were being used (which
and how much). That 1966 design document, when compared with the
current SMF implementation, will be shown to be remarkably accurate to
the needs of users even today! The presentation will also discuss the
never-announced and never-released IEHMAN report package! This
presentation is based on recently de-classified IBM design documents and
interviews with the original designers and developers of the SMF
product.
CONTENTS
A. Jun 15, 1964 SORC Report 46-47
B. 1964-1967 Brockish Interview Notes 48-49
C. Aug 25, 1967 Task Force Report 50
D. Nov 1, 1967 IBM Memo 51
E. Nov 1, 1966 Objectives 52-57
F. Mar 13, 1967 Addendum I Subset 2 58-59
G. Mar 14, 1967 Addendum II 60
H. Jul 27, 1967 Proposal Memo 60
I. 1967-1969 Schiffman Interview Notes 61-62
J. Oct 16, 1968 Subset I Final 62
K. Jan 31, 1969 Subset II Final 63-66
L. Jun 25, 1969 Subset II Final Final 67
M. IEHMAN 68
N. Jul, 1969 Release 18 69
O. Oct, 1969 Release 18.6 70
P. Jun, 1970 Release 19 71
Q. Feb, 1971 Release 20 72
R. Aug, 1972 Release 21.6 72
S. 1972-1974 Merrill Notes 73-74
T. 1975-1977 Hankison Interview 74
U. Aug 1, 1977 Objectives 75
V. Author's summary 76
W. Contributors to this paper 76
Copyright (C) 1989 Merrill Consultants, Dallas, Texas. This paper will
be published in a 1989 issue of the "Technical Newsletter for Users of
MXG". Permission is hereby granted to SHARE, Inc. to publish this
presentation in SHARE Proceedings. Merrill Consultants retains its
right to distribute copies of this presentation to whomever it chooses.
On April 7, 1964, IBM Announced OS/360. Were you computer literate then?
The IBM Design of SMF Was The Direct Result Of The 1964 SHARE Systems
Objectives and Requirements Committee "SORC". The SORC Report was
produced only two months after OS/360 announcement!
------------------------------------------------
APPENDIX F
Report of SHARE Systems
Objectives and Requirements Committee
June 15, 1964
------------------------------------------------
G.E. Bryan, Chairman L. Cohn
P.A. Cramer R. Gillespie
G.H. Mealy C.H. Weisert
"IV. JOB ACCOUNTING AND SYSTEM PERFORMANCE MEASUREMENT
The advent of multi-programmed and multi-processor machine
configurations has re-emphasized the always present need for job
accounting and made even more important the much neglected area of
machine and program performance measurement. Operating systems for
System/360 must provide as a standard feature a job accounting system
which retains records useful for both ordinary cost allocation of
machine component time and measurement of hardware and software
performance.
Accounting and statistical information should be carried in the system
on a job basis and identified by the following information supplied by
the submitter of the job:
1. A job number.
2. A programmer identification number.
3. An identification specific to the run.
4. Priority.
5. Other information as deemed necessary by
the individual installation.
In addition, in order to facilitate automatic scheduling of jobs for
optimum performance, the following should be supplied either by the job
submitter or, for "normal cases," be defined by installation standard
parameters.
6. Expected execution time, cutoff time.
7. Expected output volume (lines, records, cards) by data set, with an
installation standard limit provided in the absence of specifically
supplied data.
An installation standard factor should be applied to these numbers for
the definition of cut-off limits. The programmer should have dynamic
access to the limits for examination and modification during execution.
In certain installations there is a need to specify 'due' time -- the
time before which the job should be completed -- and 'drop' time -- the
time at which the job should be deleted if its execution has not been
started."
"The system should normally add to the job accounting record the
following information:
8. Date, and beginning clock time of the run.
9. Processor identification.
It should also add to the accounting record information for each task
performed in the job's execution:
10. Task Type (FORTRAN,COBOL,SYSTEM,EXECUTION,MESSAGE).
11. Mainframe time.
12. Equipment used (tapes,drums,data lines)
13. Output volumes (print lines, cards)
14. Core used for code.
15. Core used for data.
16. Operator intervention time (waiting for operator action).
At job completion time, all of the accumulated information should be
summarized and totaled for the user, should be added to a daily total
record, and should be output in some permanent form for later accounting
and statistical analysis.
Accounting for message switching should be done using the same
accounting mechanism with each message transmission treated as a single
task. The descriptive parameters in 1-5 are supplied by the system on
the basis of the called and calling parties. The task type (item 10) is
"message".
Flexibility should be included for the expansion of the kind and number
of statistics retained at installation or IBM option. Special
installation requirements and special IBM performance measurements
should be readily accomplised by addition of code and/or parameter
changes."
In June, 1966, Bob Brockish joined IBM.
Bob had been on the SHARE Executive Board 1963-64, and had taken Share
Systems Division job when George Mealy resigned to join IBM. Had been
at Martin-Marietta (1959) and was now data center manager at Thiokol,
Utah.
In 1966, OS/360 SYS1.ACCT accounting consisted of only an IEFACTRT Exit
which was taken only at Step Initiation (=8), Step Termination (=12), or
Job Termination (=16).
When the exit was taken, there were pointers provided to:
Job Name Step Name
Programmer Name Account Fields
Job Running Time Step Running Time
Step Number Cancel Bit
but the user would then have to format and write the data to SYS1.ACCT
using ASM code in IEFWAD. SYS1.ACCT was supported under OS/360:
PCP - 18K, 44K, and 100K Scheduler
MFT - 30K and 44K Scheduler
MVT - Scheduler
Clearly, this approach to accounting was inadequate.
Notes from my 1989 visit with Bob Brockish:
IBM'S MOTIVATION FOR DESIGNING SMF:
1. User need to account time and resource usage.
2. IBM's need to know about how the system was being used, especially
about which IBM Programs were used how much/often.
A "SYSOUT Project" had already been started inside IBM. Originally the
idea was to solicit customers to submit their SYSOUT on tape, which
would then be analyzed after the fact (for compiler messages, link
editor, etc.) to count program usage!
Ken Brownell was the manager with one other when Bob joined group.
Bob's task was to look at OS to see how data could be collected in the
operating system. IBM Programming Systems Division at Pok had just
formalized "Programming Objectives" in January. SMF was the first
project to comply! Bob began to develop SMF objectives from SORC.
The name, System Management Facility, was picked in a brainstorming
session July 6, 1966.
PROPOSED PHILOSOPHY:
IBM would provide a way to collect data, the customer must process and
report. IBM could not see a general way to report on the collected
data. Based on the SORC Report, Bob began to define the data to be
collected. The initial design was reviewed with the non-disclosure
signers at the Aug, 1966 SHARE in Toronto (but Bob could not go - Ken
Brownell wanted to see Toronto!)
People at that August, 1966 meeting:
Lora Steele, IBM SDD Share Liason
Arnold Smith, IBM Overall Liason to SHARE.
Omar Smith, Rocketdyne?
Phil Kramer, SDC Bill Thunderbirkk, ?
Roy Dickson, Okla, Phillips Petroleum Harvey Siegel, ?
Joseph E. Kelly, SOCONY Mobile Irv Lester, North American
John Noerr, Sinclair Oil
Follow up copy of the objectives was sent to:
Channing Jackson (SHARE)
Earl Althoff, VP Guide
Leroy J. Rodgers, Chair Guide COBOL project
IBM'ers also involved along the way:
Neil Lewis - primary planner at Pok Don Ludlow - SUPERZAP author
Harry Reinstein Ira Heiney
Hank Coon Bob Levy
SMF ultimately hit 160 OS modules!
The design had planned for "packages" of information, selectable,
perhaps incrementally delivered by IBM. As there was concern for
potential impact of SMF on the system, packages must allow installation
to specify only what it needed. The first package proposed, presented
at SHARE non-disclosure meeting in August, 1966, was the Basic
Accounting information (Start/Stop Times). This first package was then
reviewed at Feb 1967 non-disclosure meeting, and Bob then gave the
presentation of the revised package. End of interview notes.
Throughout much of 1967, a joint SHARE/GUIDE System Management
Facilities Study Group met under non-disclosure. At the Monday May 22,
1967, meeting in New York City at the Americana Hotel included:
Edward Boback (GUIDE) Logistics Research
T. Todd Brown (GUIDE) Iowa State University
Joseph C. Kelly (SHARE) Mobil Oil Corp.
Edward G. Laski (GUIDE) Aetna Life
John M. Noerr (SHARE) Sinclair Oil
David R. Paul (GUIDE) Iowa State University
Harvey Sigal (SHARE) Union Carbide
IBM: Paul Bouche, Poughkeepsie
Ken Brownnell, Boulder
Arnold P. Smith, White Plains
Don Miles
------------------------------------------------
This task force reported their conclusions
in the August 25, 1967 SHARE SSD:
------------------------------------------------
"Attached is a report for SSD describing user requirements for systems
management facilities under OS/360. This report contains the
information that was gathered by the Systems Management Facilities Task
Force in conjunction with GUIDE and IBM. The Task Force is now working
on further definition and input to help provide guidance to IBM as to
the OS/360 users requirements for System Management. Among other
things, the group would like input from users on what kind of system
management facilities they are currently supporting and what level of
degradation (core, time, DASD, space, devices) they consider acceptable
for each functional feature required in advanced system management.
This input should be sent to both:
C. Channing Jackson (Task Force)
R. Cleveland (IBM Poughkeepsie)
User members of the task force included
J.M. Noerr (SHARE, signer of report)
E.G. Laski (GUIDE)
C. Weisert (SHARE Sys. Division)
J.J. Jones (OS/360 Proj.)
J. Kelly (MCB)
A final comment on the task force itself. IBM has appeared quite
interested in this work and has done considerable leg work for us and we
feel that these people deserve recognition and thanks for their
exceptional devotion to getting a job done despite the task force's
sometimes apathetic reaction to their questions. These people are (all
SDD):
Paul Bouche
Bob Brockish
Ken Brownell
Don Miles
and others to a lesser degree."
------------------------------------------------
An IBM memo from Lora Steele, IBM User Group
Liason Poughkeepsie November 1, 1967 requests
IBM to make a commitment to announce SMF, and
details the history of user group involvement:
------------------------------------------------
"There has been long and consistent pressure from user groups for IBM to
provide System Management Facilities. It has become a ritual for SHARE
to request an IBM presentation on this topic every general meeting.
Although IBM has been unable to comply to date, we should plan to
provide such a presentation for the SHARE and GUIDE general meetings
following IBM announcement.
The first disclosure of IBM Confidential Information relating to SMF was
made to user group officials in August 1966 and consisted of a
preliminary version of IBM's internal objectives. These objectives were
a result of considerable effort by Messrs. Ken Brownell and Bob
Brockish of Boulder to obtain and organize the many and varied
requirements of user group members.
Mr. Brownell was assigned SDD responsibility for these objectives by
the OS Architecture and Planning management. Messrs. Don Warren and
Ward Lambert of DP were present at the first IBM Confidential meeting
held in Toronto on August 2, 1966. A revised version of the objectives
was mailed to user group officials on January 6, 1967.
In February and March, 1967 there was little activity. In April,
however, a new joint SHARE/GUIDE committee was formed to again review
the user group requirements. IBM's internal specifications were
disclosed to the committee in order to verify that their specifications
did in fact meet stated user requirements. On September 15, 1967, the
User Group Nondisclosure Agreements for committee members expired....
If IBM could make any kind of commitment for Systems Mangement
Facilities in the near future, the user groups would be mollified. If
dates and technical details are included in the announcement, they would
be pleased. If no technical details are provided, I suspect that the
committee will be revitalized and members will insist that they be
allowed to monitor IBM's implementation phase to make certain that the
considerable user group efforts have not been wasted."
IBM's earliest SMF objectives were specified in:
------------------------------------------------
From S/360-OP-001-01-Pok dtd Nov 4, 1966:
Programming Objectives
IBM System/360
System Management Facilities in OS/360
Dated: November 1, 1966
------------------------------------------------
"1.1 Purpose
1.1.2 The operating information has two primary purposes. First,
information is required to determine and equitably distribute the
costs associated with each program's use of the equipment.
Second, the installation manager requires certain information to
evaluate the performance of his installation both in regards to
his own standards as well as in comparison with other similar
computer operations in order to increase the efficiency of use of
his current equipment, improve the service provided by his
installation and plan future equipment, programming systems and
personnel requirements.
1.1.3 The dynamic control capability is required by computer center
management to monitor the work load of the System/360 as it is
being processed to verify that work to be accomplished is
appropriately submitted with proper identifying data and that
processing is carried out in accordance with accepted installation
practices.
1.1.5 For purposes of these objectives the term "user" refers to a
computer installation manager rather than to individuals using the
computer. The latter are referred to as problem programs.
1.2 Scope
1.2.1 The scope of a Systems Management Facility encompasses four
categories of support.
a. Systems Management Data Set
b. Data Collection Packages
c. Dynamic Control Entries
d. System Management Utilities Programs
1.2.2 The System Management Data Set is a permanently opened data set
specifically designed for the recording of Systems Management
Data.
1.2.3 Data Collection Packages include the gathering and retention of
data elements required for control, evaluation and cost allocation
of system usage. Items such as CPU time, storage utilization, I/O
device allocation and software components used are indicative of
the type of data required.
1.2.4 Dynamic Control Entries allow the user timely assurance of
efficient systems utilization by transferring control to a user
routine at appropriate times and places. These Entries transfer
control to user supplied coding which performs specific auditing
functions unique to each user."
"3.1.1 The required data elements are stratified into five packages as
follows:
1. Basic Job Data
2. Basic Step Data
3. Additional Job and Step Data
4. Periodic Job and Step Data
5. Sampled System Data
3.1.2 Package 1. Basic Job Data.
1. Job Log Number
2. //JOB Statement after validation
3. Entry Time of Day
4. Exit Time of Day
5. Effective Priority
6. Output Quantities - Sysout Lines/cards
7. Job Time
8. Job Termination Status
Two messages to be added to SYSOUT:
Sign on
Sign off
3.1.3 Package 2. Basic Step Data.
1. //EXEC Statement after validation
2. Step Time
3. Unit Addresses by Type of Device
4. Output Quantities (Lines/Cards)
5. Input Quantity (Card images only)
6. Step Log Number
7. Step Termination Status
One message to be added to SYSOUT:
Step Termination, Program, Time, etc.
3.1.4 Package 3. Additional Job and Step Data
Additional data elements for use in performing more sophisticated
costing and in managing data libraries.
1. Job Log Number
2. SYSIN Reader Identification
3. Reader/Intrepreter Time
4. SYSOUT Writer Class
5. Output/Writer Time
Additional per step data
6. Kept Data Set Identification
7. Deleted Data Set Identification
8. Created Private Data Volume Id (Released Private Data Volume
ID may be supported via Data Set Accounting)
9. Actual Maximum Core Used (Not the Region Parameter)."
"3.1.5 Package 4. Periodic Job and Step Data
Includes those additional items which are required for
configuration planning, software evaluation and system performance
appraisal.
1. Job Input Queue Time
2. Job Output Queue Time
Additional per step data
3. Step Start Time
4. Program Time
5. Data Set Descriptions
6. DD Statements
7. Number of Devices Used by type/step
8. Device Use Time by Type
9. Rolled Out Time
10. Storage Rolled Out
11. Number of Roll Outs.
Controlled by Operator Command.
3.1.6 Package 5. Sampled System Data
Provides the elements necessary to develop a statistical profile
of an installation's system use characteristics. After the option
is exercises at system generation time this set of data elements
is collected on a time based sampling interval (delta t).
Whenever delta t is satisfied the data will be recorded on
SYS1.MAN.
1. Time of day
2. Total Memory in Use
3. Number of Active Jobs
4. Number of Passive Jobs
5. Number of Devices in Use by type/chan
6. Number of Readers in Use.
7. Systems Work space in Use
8. Work Input Queue Lengths
9. Work Output Queue Lengths
10. Channel Queue Lengths
11. Seek Queue Lengths
12. Number of Active Tasks
13. Timer Queue Length
14. Number of Writers in Use"
"3.2 Systems Management Data Set
3.2.4 Systems Management Data will be recorded sequentially on SYS1.MAN
as the data is acquired. This method of collecting will require
the following considerations.
1. SYS1.MAN data may have to be sorted by Log Number prior to
input to analysis programs.
2. Each dumping of SYS1.MAN will be an incomplete segment of the
Systems Management Data Set.
3. A complete Systems Management Data Set contains all segments
collected between an IPL and the subsequent "drying up" of
the system.
4. In the event of an abnormal system termination contents of
SYS1.MAN shall be preserved. "
3.4 Dynamic Control Provisions.
3.4.1 Dynamic Control facilitiesj shall be provided in the form of
entries to a user routine taken at ....
3.4.2 JOB, STEP, and DD CONTROL CARD VALIDATION ENTRY ....
3.4.3 JOB INITIATION ENTRY ....
3.4.4 STEP INITIATION ENTRY ....
3.4.5 STEP TERMINATION ENTRY ....
3.4.6 JOB TERMINATION ENTRY ....
3.4.7 TIME LIMIT ENTRY ....
3.4.8 OUTPUT LIMIT ENTRY .... "
"3.6 Systems Management Utility Programs
Two Types of utility programs required in conjunction with Systems
Management Facilities are analysis programs and a list program.
3.6.1 Systems Management Analysis Programs
The purpose of these utility programs is to produce reports based
upon data contained in the SYS1.MAN data set. These reports will
provide management with the information necessary to understand
system usage and plan future improvements.
The analysis programs will yield information that describes the
workload and system utilization from several points of view.
One class of information should pertain to the overall system
usage and status independent of particular programs or job types.
Such information constitutes a "System Profile".
The "Job Stream Profile" on the other hand should describe system
utilization in terms of the characteristics of the input
workload."
A further classification of information should provide a "Job
Profile" devoted to revealing the nature of both the workload and
systems utilization in terms of the characteristics of individual
job types.
At the lowest level of detail, a "Job Step Profile" describes the
characteristics of individual job steps and their contribution
to systems resources utilization.
3.6.2 System Management List Programs
In addition to the programs to analyse System Management Data, a
utility program for listing System Management Data Sets is
required. The utility's function is to provide a printout of raw
Systems Management Data in either its natural order or in Job/Step
Log Number sequence. Operating under OS/360 this utility uses a
input either a disk or tape dump of a SYS1.MAN data set. It shall
have the ability to concatenate multiple SYS1.MAN data sets and
print out a composite listing. Information from the data set
labels will be printed at the beginning of the listing."
"4. Performance Objectives
4.1 The ultimate purpose for Systems Management Facilities is to supply
information from which installation management can know and
improve computing systems performance. Through the proper
utilization of good systems management data the user can reduce
operating costs. On this basis users are willing to sacrifice
some amount of performance for the ability to gather this data.
The system performance degradation however, cannot exceed the
reasonable value of the facilities. To guide in implementing
System Management Facilities in OS/360 the following performance
objectives are set forth.
4.2 There is to be no performance degredation when none of the Systems
Management Facilities are employed."
4.3 The performance of the operating system should not be degraded more
than 3% when the SYS1.MAN data set, Dynamic Control Entries, and
Packages 1, 2, and 3 are activated. In addition, the collection
of data in Package 4 should not decrease performance more than 4%
during those periods when it is active. Package 4 should cause no
degradation when it has been selected at system generation but is
not in use. Since the collection of Package 5 is based on the
value of a time interval. measurement of its performance should
be aimed at a time per sample basis. Each occurrence of sampling
of Package 5 data should not require more than 500ms on a 360
Model 50."
4.4 These performance objectives for OS/360 are summarized below:
Facility Enabled Timing Target Additional core
Requirements,
Maximum Bytes
none 0 0
SYS1.MAN +Dynamic Control 200ms/Job 256 + 128/Job
Package 1 500ms/Job 1024
Package 2 250ms/Step 512
Package 3 250ms/Job 512
+250ms/Step
+250ms/Data Set
Package 4 200ms/Job 1024
+300ms/Step
+500ms/Data Set
Package 5 500ms/Sample 1024
Based on jobs averaging 3.5 min and containing an average of 3 job
steps each having 5 data sets.
Based on CPU Model 50H with 2311 disk for system, SYS1.MAN and
SYSOUT residence."
Four months later, SMF Subset 2 was specified:
------------------------------------------------
Externally dated April 18, 1967
Addendum I
S/360-OP-0001-02-Bldr
Programming Objectives
IBM Operating System/360
Data Set Management & DA Space Accounting
Dated Internally: March 13, 1967
------------------------------------------------
"This addendum extends the scope of the System Management Facilities to
supply the user with the tools required to manage data set libraries and
to control and account for space usage on direct access devices. The
facilities to furnish this capability include the recording of data set
records on SYS1.MAN and a utility routine to retrieve the data set
records, put them in a data set management file, produce a data set
inventory and maintain the data set management file.
New data will be recorded when Package 3 is selected:"
A. NEW, Non-temporary Data Sets
1. Identify as NEW
2. Data Set Name (including GOOOVOO)
3. Creation Date
4. Expiration Date
5. Device Type
6. Number of Volumes
7. Volume Serial Numbers
8. Public or Private
9. Shared or Exclusive
10. Number of accesses to read
11. Number of accesses to write
for direct access for tape
12. File organization Data set sequence Nr
13. Amount of space Type of labels
14. Unit of Space Recording density
15. Number of extents 7 or 9 track recording
B. OLD Data sets - similar to NEW
C. Temporary Data Sets
1. Identify as Temporary
2. Device Type
3. Number of temporary data sets
4. Total number of volumes involved
5. Total number of accesses/records.
6. Total amount of space in bytes"
And then, amazingly, Subset 2 described the:
"IV. Data Set Management Utility
The Data Set Management Utility is intended to run daily or periodically
as required by the installation to provide information to guide the
operations staff in purging the data set library and to maintain the
circulation of data volumes.
A. Functions of the Routine
1. Extract data set records from concatenated SYS1.MAN dumps and use
them to build and maintain the Data Set Management File. This
Utility does not physically remove data set records from the Data
Set Management file on this basis. Instead it "deletes" by setting
a flag and inserting a deletion data in the appropriate record.
Actual removal of data set entries will be made via punched card
input after the information has been used for billing, etc., by the
user.
2. Produce a report by device type and volume serial number of data
sets whose expiration date has elapsed. This listing includes Data
Set Name, Programmer's Name, Accounting Fields, Creation Date,
Expiration Date, Date Last Written and Date Last Read."
3. Produce data set inventory report by device type showing activity.
This listing includes Data Set Name, Programmer's Name, Accounting
Fields, etc....
4. Accept, as input, punched cards with deletions and changes to data
set records in the Data Set Management File. These cards will be
prepared by the user's data librarian and input into the file
maintenance run. Through these cards, data set entries can
actually be removed from the file and revisions can be made to
accounting fields, responsible programmer's name, etc.
5. Provide exits for user supplied coding to perform other functions
desired by the installation."
Addendum II was dated a day later:
------------------------------------------------
S/360-OP-0001-03-Bldr
Programming Objectives
IBM Operating System/360
Job and Step Timing
Space Accounting
Internally Dated: March 14, 1967
------------------------------------------------
"This addendum replaces "Job Time" in the original specification with
"Job CPU Time", and adds a new element:
7b. Job Unoverlapped I/O Time
Job Unoverlapped I/O Time is the accumulation of time during which both
of the following conditions are satisfied:
a) input and/or output activities are being carried on by or for a job
b) and that job is not using the central processing unit.
Job CPU Time plus Job Unoverlapped Time equals the time a job would
require if it was the sole inhabitant of the computer system. It is the
summation of these two time measurements that ... should be compared
with when determining if the job is exceeding its maximum running time."
------------------------------------------------
Dated 7/27/67 is a Proposal for Extensions
to OS/360
------------------------------------------------
Proposing:
- Maintenance of the DA Volume Inventory
- A New Volume Reorganization Utility
- A New PDS reorg-needed status message
- A New PDS reorganization utility!
It appears this document was not accepted, but it contains a tantalizing
bit of data:
"3. Market Requirements
3.1 The problem of DA space deterioration potentially affects each of
the estimated 550 installations using OS as their primary
operating system. Also to be considered are the additional 1596
forecasted future users of the system."
550+1596 = 2146 Future OS Customers!!!
Initial specifications were completed in Summer, 1967 and SMF moved to
Poughkeepsie (Pok) for implementation under Saul S. Schiffman, with
Lynn Weisenreider in his group. Notes from 89 Saul Schiffman interview:
Biggest implementation difficulties:
to convince Supervisor in IOS to put in lines of code in Operating
System - they were very conservative on instructions in path.
Testing together to see that it worked was new, OS was the first large
scale software project.
Had to firmly make rules how to access SMF. Some developers did not
want user records - could garbage up the collection, but they lost.
MANX/MANY had to be handled by the system
Very hard at the beginning, everybody had their own ideas.
"Hundreds" of pieces of information.
Rule: Unless you can show me how you will use it, I won't add it.
That reduced hundreds to 30-40 items.
Complaint of expected SMF overhead of 5% was answered by jury rigging
SYS1.ACCT for all of the same data. Discovered that it too took 5%,
and showed in-line capture no worse than exits.
This convinced IBM that measurement could be an integral part of the
operating system, and also showed that SMF data was not optional - a
site simply could not run without this data.
Was the customer Measurement or Accounting?
Accountants - look at this data
Performance Measurement - look at that data
Picked common ground between the two.
Made attempt to convince users to use only the repeatable fields, and
make data more repeatable - MFT easier than MVT.
Wanted one project for billing and for measurement and evaluation.
When conflicts arose, measurement & evaluation guys gave up on
additional data fields.
Accounting - some things are outside of control (eg paging) - could
not predict branching impact, thus cannot charge for it. Began to
look at counting from application.
IBM could not answer "How do you account?"
"We were not going to write accounting and
billing routines at IBM.
Instead, we will document and provide data"
SMF Implementation began at the start of 1968. PCP, Fortran, MFT I, MFT
II development were at Kingston, while MVT, IOS, SVS, MVM were all at
Poughkeepsie. MVM (Multiple Virtual Memories) became MVS.
SMF Implementation Options now considered:
1. Have a program in the system dynamically sample and adjust the
system based on the actual measurements.
- heuristic approach
- radical (remember, this WAS 1968!)
- major proponent of this option was
Bart Page -"Brain behind the SRM.
Mind beyond others.
Ahead of his time, but
Not a good salesman."
or, what we actually ended up with:
2. Extension of traditional SMF design
Not much on analysis
Minimal reporting
End of 1989 Interview notes.
On October 16, 1968, Saul's group distributed their Final Specifications
on Subset 1:
------------------------------------------------
System Management Facilities (Subset 1)
Final Functional Programming Specifications
(MVT)
OS/360-OS-0043-05-POK
------------------------------------------------
Unfortunately, I have not see a copy of that document. Based on the
final implementation, however, Subset 1 created the following SMF
records:
Type 0 - IPL Record
Type 1 - Wait Time Record
Type 2 - Dump Header Record
Type 3 - Dump Trailer Record
Type 4 - Step Termination Record
Type 5 - Job Termination Record
Type 6 - Output Writer Record
Type 7 - Data Lost
Type 8 - I/O Configuration
Type 9 - VARY ONLINE Record
Type 10 - Allocation Recovery Record
Type 11 - VARY OFFLINE Record
Type 12 - End-of-Day Record
Earlier, on February 7, 1968 there had been:
------------------------------------------------
Data Set Management and D. A. Space
Accounting (SMF Subset 2)
Initial Programming Functional
Specifications
S/360-OS-0068-00-POK
------------------------------------------------
which I have not seen either. However, in January 31, 1969, the "Final"
for Subset 2 was published:
------------------------------------------------
S/360-OS-1010-00-Pok
Final Programming Functional Specifications
IBM OS/360
Data Set Management and D.A. Space
Accounting (Subset 2)
Dated January 31, 1969.
------------------------------------------------
This identified the new SMF records that would be added by (Subset 2):
"2.1.4.2 Record Types
Type 14 & 15 - Non-Temporary User Data Set
Type 16 - Temporary User Data Set
Type 17 & 18 - Data Set Status SCRATCH/RENAME
Type 19 - Direct Access Volume Dismount
Type 20 - Job Commencement"
This "Final" Document also described three new specifications in
significant detail:
Direct Access Inventory
Tape Inventory
Resource Management Utility
Highlights from this IBM analysis package description then go on:
"2.1.5 The Resource Management Utility
The Resource Management Utility collects volume and data set information
from the SMF data set. This consolidated data is available in report
form. The main functions of the utility are as follows:
A. It uses volume and data set information from the SMF data set to
create and update records in the inventory data sets. There is one
inventory for direct access resources and one for tape.
B. From volume information in the inventories it produces a volume
inventory report and SCRATCH volume reports.
C. From data set information in the inventory it produces a variety of
data set reports. The data set inventory records are selected and
arranged according to such parameters as data set name, volume
identification, account number, expiration date and last usage date.
D. It creates control cards for IEHPROGM to SCRATCH data sets which
have expired.
E. It creates control cards for the Resource Management Utility to
REMOVE data set inventory records from the inventories for data sets
which have been SCRATCHed or DELETEd."
"F. It accepts control cards to remove volume and data set information
from the inventories.
G. It can compress the inventories.
Each inventory is a partitioned data set (PDS). Each directory entry in
the PDS reflects a particular volume. The member itself contains
information about data sets on that volume.
This was followed by twelve pages identifying each field in each
inventory record and its source SMF record (5,14,15,17,18,19, or 20)."
and then the JCL:
"2.1.5.6 Job Control Language and Control Statements Used by the Utility
The Resource Management Utility can be invoked by the following job
control language
//jobname JOB positional parameters and
keywords
//stepname EXEC PGM=utility name
//SYSPRINT DD parameters describing
SYSOUT device
//SYSDATIN DD parameters describing direct
access inventory PDS
//SYSTAINV DD parameters describing tape
inventory PDS
//SYSMAN DD parameters describing the
SMF data set
//SYSREPRT DD parameters describing report
device
//SYSPUNCH DD parameters describing punch
data set
//SYSIN DD parameters describing control
statement data set "
"The control statement data set (described on the SYSIN DD statement)
contains one or more of the following operations:
A. UPDATE
B. REPORT
C. REMOVE
D. COMPRESS"
Then followed twelve pages detail the syntax and use of the many
operands for each operation.
The sixteen error messages produced by the Resource Management Utility,
IEH901E through IEH916I are then documented, and give the first clue as
to the planned name for the Resource Management Utility program name of
IEHMAN!
"2.3.2.2 Resource Management Utility Requirements
The utility program is designed to be in a planned overlay structure
with a minimum of 15K required for executable code at one time. In
addition, core storage is required for buffers. The number of bytes
needed for buffers is device dependent. Maximum buffer sizes (in bytes)
are shown in the table below:
Device Type Maximum buffer size
2301 Drum 21,000
2302 Drum 5,400
2303 Drum 5,300
2311 Disk 4,000
2314 Disk 7,900
2321 Data Cell 2,400"
And even a performance evaluation of the utility:
"The direct access inventory has entries for ten volumes, each with nine
extensions, giving a total of 100 members in the directory. Each member
is 3500 bytes long and contains 15 data set inventory records, giving a
total of 1500 data set entries.
An UPDATE is estimated to take 9 seconds.
A REPORT VOLID operation, which produces a list of direct access volume
inventory information is estimated to take 1 second.
A REPORT DS operation, which produces an unordered list of all data sets
on the printer, is estimated to take 82 seconds. Further time would be
required to produce a sorted list.
A prototype of the utility was coded and tested in several different
versions. MVTTRACE was used to obtain timings of the final version
described below.
SYS1.MAN data set resided on a 9-track 2400 tape drive, model 3, and was
recorded at 800 bpi. The direct access inventory resided on a 2311
device with a member blocksize of 3200 bytes. The printer was a 1403
device with 120 print positions. The CPU was a model 50, the system
MVT, Release 14."
"The first UPDATE run required 12.2 minutes. This initialized the
inventory data sets.
A. The input stream from SYS1.MAN contained the following records
arranged as if coming from an MVT environment:
60 job commencement (type 20)
60 job termination (type 5)
1500 old non-temporary (type 14)
1500 new non-temporary (type 15)
60 temporary (type 16)
60 SCRATCH (type 17)
20 RENAME (type 18)
80 direct access volume dismount (type 19)
The resulting direct access inventory contained information about 1862
data sets, of which 39 had been SCRATCHed. There were 197 directory
names in the directory, of which 148 were base members and 49 were
extension members. The average number of data sets per volume was 12.6.
The average length of data set inventory records was 208 bytes.
B. The second UPDATE required 19.7 minutes."
In a separate section of the Final Specification the error expectations
of SMF were predicted:
"6.4.2.1 APAR records for IEBPRTPCH, a utility program estimated to be
of a similar size (13,000 bytes) were used to project expected
errors and severity.
6.4.2.3 All Severity 1 errors should be resolved before integration.
Only two OS utility programs have ever received Severity 1
APAR's after release.
and the accompanying table showed
After Customer Ship
Expected: 6 months 18 months
Severity 2, 3, 4 errors 7 15
Man hours to correct 40 40
Machine hours to correct 5 5"
And then along came the Final "Final":
---------------------------------------------
S/360-OS-1010-01-Pok
Final Programming Functional Specifications
IBM Operating System/360
Data Set Management and D.A. Space
Accounting (Subset 2)
Internally Dated June 25, 1969.
---------------------------------------------
This document was much thinner than its predecessor, and states on its
cover letter:
"This revision of the referenced specification contains modifications in
the following major areas.
1. The Resource Management Utility program description has been deleted.
THUS DIED SMF Reporting.
This final specification also eliminated the type 16 (temporary data
set) by redesigning the type 14 and 15 records to what we now have.
In addition, design specifications that have departed from the initial
specifications were identified:
"1.3.2 From Initial Specifications: These specifications depart from
the Objectives and Initial Specifications in the following respects:
a. No I/O device timing will be performed.
c. The TCT of Subset 1 will be expanded to include information
previously intended for a new control block (the IOCT). This will
eliminate the need to modify the DEB."
The final error table was now modified
" After Customer Ship
Expected: 6 months 18 months
Severity 2, 3, 4 errors 5 (was 7) 20 (was 15)
Man hours required 4 (was 40) 10 (was 40)
Machine hours required 4 (was 5) 10 (was 5)"
"Systems Management Utility - IEHMAN"
The IEHMAN Planning Guide describes the same features that were
specified in the SMF design for Subset 2, called the "Resource
Management Utility"!
Not only did the IEHMAN Planning Guide exist within IBM, but through an
error, a small number of copies were actually shipped from
Mechanicsburg, Pa., to some IBM customer sites!
Once the error was discovered, IBM immediately recalled all copies!
Ken Plambeck was the author of the IEHMAN package.
The project had 10-12 people 12 hrs/day, but the product was killed when
the manager could not get sponsors.
Even though IEHMAN was never announced nor ever released, it showed that
IBM designers, even in 1966-1967, knew that analysis tools would be
needed to exploit the SMF data.
------------------------------------------------
In May, 1969, C28-6712-0, Planning for System
Mangement Facilities (62 pages) described
the version of SMF planned for Release 18.
------------------------------------------------
but the real announcement of availability was:
------------------------------------------------
IBM System 360/Operating System
Release 18 Guide
GC28-6718-1
July, 1969
------------------------------------------------
"System Management Facilities
SMF is a new feature of the operating system, selectable at system
generation in conjunction with the MVT configuration. SMF may not
be specified in conjunction with Model 65 Multiprocessing.
SMF provides two basic functions:
Collecting and recording system-, job-, and step-related data for each
job processed by the system.
Monitoring job processing via exits from the control program to
installation-provided routines at specific points during the
processing of each job."
Data Collection: SMF writes 13 types of records to a data set resident
on either tape or direct access. The records contain such information
as: system configuration, job and job step identification, CPU wait
time, CPU usage by each job and job step, and I/O device usage and main
storage usage by each job step. The writing of SMF records may be
supressed at IPL. An installation-provided routine can periodically
analyze the SMF dataset to evaluate system efficiency, performance, and
usage.
Job Monitoring: SMF provides five exits within the control program to
installation routines:
From the reader/interpreter of the job just before each job control
statement is interpreted.
From the initiator/terminator when a job is selected for initiation.
From the initiator/terminator when a step is selected for initiation.
From the initiator/terminator when a step and/or job is terminated.
From the timer second level interruption handler (SLIH) if a specified
CPU or wait time limit for a job or step is exceeded."
If no wait time limit is specified, the default value provided by IBM
is 15 minutes; in previous releases the default value was 30 minutes.
A new macro instruction (SMFWTM) may be used in exit routines to write
records to the SMF data set. A test procedure (TESTEXIT) is provided in
SYS1.SAMPLIB to facilitate routine testing.
You must also provide analysis and report routines to process the SMF
data set."
------------------------------------------------
IBM System/360 Operating System
Consolidated Document
Release 18
GY28-6681-2
Third Edition (October, 1969)
------------------------------------------------
"Updated Version of Release 18, designated as Release 18.6, may now be
ordered from PID.
Approximately 40 PTFs have been applied to the distribution libraries,
correcting more than 80 APARs.
(Release 18.0 text:)
A second release of SMF will support MFT, M65 Multiprocessing and Remote
Job Entry.
Graphics Job Processing (originally planned for the second release of
SMF) is supported in this initial release of SMF.
Primary Control Program (PCP) and Conversational Remote Job Entry (CRJE)
are not supported.
Performance
If SMF is chosen at System Generation, performance will be reduced.
The performance reduction will be dependent upon the installation's use
of the following:
. SMF buffer size
. Device used for the SMF data set
. SMF data set allocation size
. Number of jobs run per day
. Execution time of the installation exits
The performance reduction to the system when all System Management
Facilities are functioning can be less than 5%.
Storage Requirements
A fixed amount of main storage is required when SMF is chosen at System
Generation. In MVT a maximum of 1500 bytes is added to the main
resident storage. Supervisor Queue Space is used for data collection
tables, new job queue entries, and the user defined SMF buffer. The
variable storage depends on the number of active jobs in the system and
the SMF options chosen."
------------------------------------------------
IBM System/360 Operating System:
Release 19 Guide
GC28-6733-1
June 1970
------------------------------------------------
Announced Subset 2 of SMF and support for MFT and M65 Multiprogramming
and RJE with all twenty-one SMF records (0-15,17-20) with one new
record,
Type 13 - MFT Partitions
and one new Exit
(IEFUSO) for SYSOUT data set limit exceeded
------------------------------------------------
System Programming Guide Release 19
------------------------------------------------
"Example of SYS1.MANX Space Requirements
ID bytes
1 IPL per day 0
20 Devices online 8,19
2 Vary Onlines per Hour 9
2 Vary Offlines per Hour 11
1 Device Allocation per Hour 10
1 Scratch Dataset per 4 Hours 17
1 Rename Dataset per 4 Hours 18
24 Accumulated Wait Time 1
Total for these records 2,025
Job Processing
1 Start of Job 20
1 End of Job 5
1 Dismount 2 Volumes per job 19
3 Steps per job 4
Step Processing
3 DD statements per step 4
2 Input datasets per step 14
2 Output datasets per step 15
2 Output writers per step 6
Total for one job 6,303
Total for 48 jobs in 4 hours 302,688
SMF headers
Record Descriptor Words 5,044
Block Descriptor Words 1,684
TOTAL SMF DATA FOR 4 HOURS: 311,411"
------------------------------------------------
IBM System/360 Operating System:
Release 20 Guide
GC28-6730-0
January 1971
------------------------------------------------
Announced 20.0 with support for S/360 and new S/370 155 processor (S/370
165 in April) and indicates to expect 20.6 in 6-8 months.
------------------------------------------------
IBM System/360 Operating System
System Management Facilities
GC28-6712-4
(Fourth Edition, February 1971)
------------------------------------------------
Applied to Release 20.1 and identifies the eleven new SMF records
created with Release 20 (now totaling 31 record types):
Type 21 - ESV Record
Type 30 - Start TSO Record
Type 31 - TIOC Initialization Record
Type 32 - Driver Record
Type 33 - Driver Modify Record
Type 34 - TSO Step Termination
Type 35 - TSO Logoff Record
Type 38 - Initial TSO Configuration Record
Type 40 - Dynamic Allocation Record (not documented!)
Type 41 - Modify TSO Record
Type 42 - Stop TSO Record
------------------------------------------------
IBM System/360 Operating System:
Release 21 Guide
GC28-6730-2
March, 1972 (Release 21.0)
------------------------------------------------
Announced the REC parameter and minor change in format of SMF records,
and indicated that 21.6 would be along in 4-6 months.
------------------------------------------------
IBM System/360 Operating System:
Release 21 Guide
GC28-6730-04
August, 1972 (Release 21.6)
------------------------------------------------
was announced, with no SMF enhancements.
In October, 1972, I first used the Statistical Analysis System, SAS, to
read SMF records from OS/360 Release 20.6 at State Farm Insurance. At
that time, SAS cost $100! By early 1973, I reported our successful
results to user groups (SAM, TESDATA) and ACM (SIGSIM, Chicago). John
Chapman demanded that I present at SHARE.
In March, 1974 (SHARE XLII, Houston), in the CME project, I presented
SMF/SAS; CME scheduled an open session for the Chicago SHARE.
That summer, IBM announced SGP, their Statistics Gathering Package,
written by Bill Tetzloff. The open session at the August, 1974, SHARE
meeting in Chicago began with Bill's SGP product description followed by
State Farm Auto's presentation of their use of SAS and SMF data. Over
750 people (half of SHARE attendance) were present! While SGP was truly
a valiant IBM effort to provide reporting from an extract of a fixed set
of SMF fields, it was not easily tailored, was cast in concrete, and
appeared inflexible. This audience then saw the SAS language for the
first time, and saw SAS actually used for productive SMF analysis. At
the end, one attendee pointedly asked IBM, "Now that you have seen SAS,
is there any reason why we should still consider SGP?"
This discovery that the SAS language was highly suited for analysis of
SMF and RMF data led to many SHARE sessions that demonstrated that SMF
data analysis really did save money and answered data center
management's questions (response, capacity, service objectives, etc.).
SAS was the tool that made SMF analysis practical, in 1974.
The early releases of MVS became available:
VS/2 Announced August 1972
?/73? MVS 2.2 MF-1 Type 70-77,79
?/75? MVS 3.0
5/76 MVS 3.7
8/76 SU7 Supervisor 2
8/76 SU20 RMF
9/76 SU11 TSO CMD. Package
9/76 SU13 TSO/VTAM
3/78 SU58 TSO/VTAM Level 2
3/78 SU50 MVS SE1 (for 3.7)
7/78 SU78 TSO Session Manager Rel. 2
3/79 SU65 MVS SE1.1 (for 3.7)
3/79 MVS 3.8 (SCP2)
8/79 SU74 MVS SE2 (for 3.8)
Type 23, 30, 32, 90
1/85 NPDA V3 R2 (Type 37)
2/85 DFSORT R7 (Type 16)
3/85 NLDM R3 370 (Type 39)
7/86 DB2 (Type 100,101,102)
By the mid 1970s, large TSO shops (several hundred concurrent logged on
users) began noting degredation due to the BSAM SMF writer:
- ENQ/DEQ was used by all SMF records, a very slow, serialized server
for every logical record written.
- TCLOSE to update VTOC after every block was written moved the disk
drive arm - the drive shook like a washing machine!
- WRITE VERIFY doubled the I/O time
In addition, the 1975 SHARE-IBM meeting in New York was called because
users were not migrating from MVT to MVS, and were blaming SMF
accounting issues, technical issues about actual numbers in the records
(eg., more absolute CPU time, the loss of "storage measurement" a la
"K-Core Hours" from MVT, the inclusion of PCI counts into SMF EXCP
buckets) caused, Ron Hankison, then IBM Representative to SHARE MVS
Group, to become the local SMF expert within IBM.
From Ron Hankison 1989 interview:
Accounting requirements from SHARE and GUIDE had built up, and nothing
had changed since the original OS implementation (VS 2.2 and VS 1.6 had
only added a few fields regarding paging and Virtual storage).
TCLOSE was out of control, the constraint was apparent and there was a
backlog of user requirements.
Project goals were to:
Fix the performance constraint
Add functionality
Restructure after 5-10 years experience
Estimated project by doubling the then-known size of SMF (Source Lines
of Code) and used that (and IBM internal estimates) for the actual
estimated man-hours.
The final implementation took four times as much as the estimate,
requiring 12 people 1.5 to 2 years for SE2 SMF Writer redesign.
Because no one else in VS 2 cared much about SMF, the development was
independent, which allowed many things to be considered and many went in
and out of the final SMF enhancements.
Primary developers were:
Barb Butler
Bill McTiernan
Steve Rosengarn
Joe Winterton
------------------------------------------------
Programming Objectives and
Initial Programming Functional Specifications
MVS Accounting Facility
August 1, 1977
------------------------------------------------
"1.4 Summary of Specifications
The rewrite of SMF will resolve the numerous, known problems with SMF.
The performance of SMF will be much improved by the elimination of
bottlenecks during the collection and writing of the data. Additional
data will be available for recording that fills in several known
exposures for resource utilization and system activities. Flexibility
will be built in that allows the installation improved control over the
recording and make tradeoffs between the integrity of the data collected
and the performance impact. Complexity will be reduced by the
capability of establishing a common file to contain all the pertinent
information needed to manage the installation. Additional useability
items will make this package very appealing to DP installations."
1.5 Summary of RAS Characteristics
Due to the critical nature of the data being handled, minimizing the
opssibility of loss of data will be stressed in the design of this
package. The improvements described in this document will be covered by
either an FRR or ESTAE routine to ensure that loss of data is held to a
minimum in the event of a failure in the SMF component. Optional
facilities will be provided to minimize loss of data due to system
failures. Programming techniques known to have demonstrated improved
quality will be used during implementation and test."
2. User Requirements Addressed
Started Task Reporting
Interval Reporting
Performance/Overhead of Data Collection
Recording Selectivity
Proliferation of Data and Records
Installation Tracking Completeness
Accounting Direction
Proliferation of Tools
7. Performance
The overall reduction of SMF overhead and its impact on system thruput
will be significant in those environments that have a high volume of
data that is recorded to SMF. The performance improvement of the
collection routine and its branch entry capability will provide a much
improved interface for those components that should be recording to SMF
but were concerned about the SMF bottlenecks.
The path length for a Write to the SMF data set is approximately 60,000
instructions. The frequency of that event is installation dependent but
probably averages about once every 13 logical records. The size and
frequency of record receipts will be increasing rapidly as processor
speeds increase. The revised path length by using the VSAM ICIP in SRB
mode is approximately 2500 instructions, resulting in a reduction of
57,500 instructions per Write."
Summary of the author's opinions:
1. SHARE was instrumental in the creation of SMF.
2. Users had a fair idea of what data was needed.
3. IBM designers in 1966 knew more than users of the range of data that
we would ultimately need.
4. The iterative process between IBM designers and IBM users provided
realistic validation of the design before implementation.
5. Designer's specifications and wants exceeded the funding and time
for implementation.
6. The 1966-1969 IBM design and implementation process was better
structured, managed, and documented than your company's most recent
managed software project in 1989.
7. Based on this example of IBM design practices of twenty years ago,
imagine the detail we would find in today's IBM design documents!
8. SMF made IBM pre-eminent in the business of real data processing by
giving DP management actual measures of the resource usage and the
service (response) for each user. DP could then "manage by objectives"
and prove to the president the value and costs of the services DP
provided the company. No other vendor of hardware and software has
provided the accurate measurements that IBM has given its customers.
9. As good as it is, it still isn't perfect:
MVS/ESA added three important CPU measures to
the type 30 (task level) record,
RCT - Region Control Task CPU
SLIH - Second Level Interrupt Handler CPU
HPT - Hiperspace CPU
but we do not have these same CPU measures
in the type 72 (performance group) record.
SHARE REQUIREMENT, ANY ONE?
Contributors:
With sincere thanks for the dedicated developers designers and users of
SMF data, I especially salute these contributors to this research:
Bob Brockish, (retired) IBM Tom Bell, Rivendel Consultants
Lynn Weissenreider, IBM Boulder Brian Currah, BDC Computer Services
Saul Schiffman, IBM Japan Aron Eisenpress, CUNY
Ron Hankison, IBM Kingston Mario Morino, Morino Associates
Dick Armstrong, IBM Gaithersburg
Jim Doyle, IBM Poughkeepsie
Jack Higgins, IBM Publications
especially for their treasure-trove of original IBM project
documentation as well as their time in reminiscing on personal
remembrances.
This section added March 4, 2017, after the article was posted to the
IBM-MAIN ListServer, and Joe Witherton (cited above) posted this note:
Thanks Barry for the walk down memory lane.
I had a few years working on SMF in my early IBM days. First I was the
receiving programmer to enhance SMF for VS2 rel 1 in Poughkeepsie when
it moved from Boulder to Pok. I did some minor SMF enhancements for VS2
rel 1.
Then later in the late 70's I also was team leader for this project to
enhance SMF and will add to this text below a little. We listened to
Barry M and Share as they helped us understand the customer needs.
Actually we knew the sizing effort had to be carefully handled at IBM
and knew how much people months we could recommend to spend on the
project and if sized correctly the project may not be approved. We low
balled the sizing. Once we got it approved we felt we could get the
needed people added to finish the project and we did. I later took the
hits for our poor sizing but we met the customer needs as the team felt
the passion to make these enhancements for Share customers. I also
presented for IBM at Share in San Francisco to a packed ballroom the
announcement for MVS SE2 and described how the release met many of the
Share requirements for SMF.
"Estimated project by doubling the then-known size of SMF (Source Lines
of Code) and used that (and IBM internal estimates) for the actual
estimated man-hours.
The final implementation took four times as much as the estimate,
requiring 12 people 1.5 to 2 years for SE2 SMF Writer redesign.
Because no one else in VS 2 cared much about SMF, the development was
independent, which allowed many things to be considered and many went in
and out of the final SMF enhancements."
Joe Winterton