System 1032
Lesser Known Features: Part 6 By Tym Stegner Looking Anew at Record Descriptors
The System 1032 Record Descriptor (RD) is the backbone of the System 1032 data transfer mechanism. An RD describes the external data read to or written from System 1032 datasets. They are part of APPEND, DUMP, and LOAD commands, as well as the System 1032 API for record transfer operations.
An RD reads data from or writes data to program record buffers or to and from files. Many RD command elements are available to facilitate the process of reading data into System 1032; some are also available for exporting data. Although many RDs may be hard at work in a System 1032 application, because they are tucked away in dataset definitions and commands, they have become lesser-known. This article intends to bring them out of hiding by revealing many of their capabilities.
Handling Data with Versatility
RDs support all the System 1032 data types, including arrays, groups, arrays of group, varying length text, and binary varying data. Data is read out and written in as text format or binary format. You can transfer binary data with scaling and precision defined, as well as using trailing overpunch formats. Text transfer supports EBCDIC and numeric formats such as hexadecimal and octal. In addition, the ALLOW MULTIPLE_RECORDS quality on text fields lets them read data from files with multi-record file formats.
RDs can also work with different physical file formats. For example, the RECORD_STREAMS option can read StreamLF format files, which are common on UNIX or PC-based data systems.
Using the qualities available for text-structured fields, System 1032 can read delimited files, such as tab-delimited, comma-delimited, or quote-and-comma delimited, although the latter requires additional definition adjustments. (For more details on handling quote-and-comma delimited files, contact us via email.
When used with a view dataset, an RD can effectively read data from multiple datasets at one time. However, this does not work for reads from input files and a single RD cannot generate multiple output files.
Creating an RD
Structurally, an RD is one or more FIELD statements with a data type defined that corresponds to one or more attributes in a dataset. The fields in the RD need not match either the sequential order or the exact data type of the attributes. Furthermore, you need not specify all the attributes in the dataset, and can include dummy fields, if required. A dataset may have several RDs defined for different applications.
Often an RD is stored directly in the dataset, thus making it available whenever the dataset is open. Alternately, the RD is stored in an application library, or executed from a DMC file, or even dynamically defined by a program according to its data transfer needs. For example, the System 1032 ODBC interface’s dictionary tool, ODBC_DICTIONARY.EXE, creates and stores a table-specific RD every time a new table is defined or modified.
Converting a Data Type from Field to Attribute or Back
System 1032 matches field data to the correctly named attribute and converts the data as required from the field data type to the attribute data type, or visa versa. Two charts in the System 1032 User’s Guide, Module 3, Chapter 2: Converting Between External and Internal Data Types, describe the permissible conversions for either text or binary input data. In broad strokes, you can convert:
An RD can recognize and write multiple date and time formats, similar to the WRITE FORMAT command capabilities. The following formats are supported for input and output.
Note: Shortening the length of a date field by two spaces changes the display of dates from a 4-digit year to a 2-digit year. Also, time formats support up to 7 digits of precision for fractional seconds.
Examining a Couple of RDs
The SHOW DAMAGE command for a dataset displays a record of possible damage that happened to a dataset. When a dataset is rebuilt (for maintenance), this damage history is lost. However, you may want to retain a history of the damage a dataset experiences. Only the first 20-odd entries are stored within the dataset; additional entries are stored in an external, binary-format file, named dataset-name.DMU. The following DAMHIST dataset definition can store damage history data, because it includes two RDs: one to read the output of the SHOW DAMAGE command, the other to import data from a DMU file.
Dataset DAMHIST Comment “Damage history for ???” Attribute EVUSER Text 12 Title “Event/Username” Keyed Attribute EVNODE Text 6 Keyed Attribute EVCMD Text 3 Title “Command/Code” Keyed Attribute EVENT Text 3 Title “Event/Code” Keyed Attribute EVERROR Integer Title “Event/Error” Format H8 Attribute EVTIME Date_Time Title “Event/Timestamp” Fmt d3.3 ! Procedure FIXUP_EVENT Var EX Array(0:9) Of Text 3 Let EX(0)=”UPD”, EX(1)=”IMP”, EX(2)=”DAM”, EX(3)=”COM”,- EX(4)=”INT”, EX(5)=”JON”, EX(6)=”JOF”, EX(7)=”JCR”,- EX(8)=”SBK”, EX(9)=”FBK”; Find EVENT Between “0” And “9” Change EVENT EX($Int(EVCODE)) End_Procedure ! RD DHTEXT Field EVUSER Text Column 1 12 Field EVNODE Text Column 18 23 Field EVCMD Text Column 27 29 Field EVENT Text Column 33 35 Field EVERROR Text Hexadecimal Column 39 46 Field EVTIME Text Date_Time Column 49 68 End_RD ! RD DHBIN Field EVTIME Binary Date_Time Length 8 Field EVENT Binary Integer Length 2 Field EVNODE Text Length 6 Field EVCMD Binary Integer Length 4 Field EVUSER Text Length 12 Field EVERROR Binary Integer Length 4 End_RD End_Dataset
Examining the RD Definitions
Compare the sequence of attributes in the dataset with the fields RDs. The binary file stores the data in a different sequence than does the output of a SHOW DAMAGE command. But an RD can ignore the sequence of attributes, because it just matches an attribute name with a field name.
The two fields, EVCMD and EVENT, are defined as Text in RD DHTEXT, and Binary Integer in the RD DHBIN. The FIXUP_EVCODE procedure locates records where those fields contain numeric strings, and converts them to the code values. The Binary Integer data type in the file is automatically converted by System1032 into a numeric string, then stored in the Text attribute.
The output of a SHOW DAMAGE command, a text file, is read with absolute column positions, whereas the binary file is read with relative field positions, based on the length of the binary data elements.
In Summary
Whether used from flat files or within programs, RDs add greatly to the functionality of System 1032. For more details on understanding the code or writing an RD, see the System 1032 Programmer’s Reference, Module 1. For additional explanation and discussion about using RDs, see the System 1032 User’s Guide Module 3, Chapter 2: Importing/Exporting Data.
Model 204
V5R1 RESTART Recovery: Major Enhancements, Part 2 By James Damon
In the August 2003 issue of CCAPRINT I discussed the automation of secondary recovery in V5R1. This month I will concentrate on significant performance improvements you can garner from RESTART recovery.
Setting the CPMAX Parameter
One performance feature that has always been available is setting the CPMAX parameter to one. This keeps the CHKPOINT dataset as small as possible and ensures that ROLL BACK processing completes in the minimum amount of time possible. The CPMAX parameter specifies the maximum number of checkpoints to keep in the CHKPOINT dataset. When CPMAX=1, only one checkpoint interval of ROLL BACK information is kept. This not only minimizes the amount of time required for ROLL BACK processing, it also minimizes the space consumed by the CHKPOINT dataset, which helps preclude a CHKPOINT-full condition that results in Model 204 termination.
CHKPOINT schematic
Figure 1. Checkpoint scheduled every 10 minutes
In Figure 1 a checkpoint was taken at 12:15 and pre-images of various pages from various files were written after the checkpoint. If CPTIME=10 and CPMAX=1, at 12:25 another checkpoint is attempted. If successful, the new checkpoint record is written at the beginning of the dataset, overwriting all previous information. This process repeats every 10 minutes:
Figure 2. System failure, power failure or other abnormal termination
If the Online terminates abnormally at 12:32 due to a system failure, power failure, or another event, ROLL BACK processing has only seven minutes (elapsed time) worth of pre-images to read and reapply to complete the ROLL BACK phase of RESTART recovery.
Figure 3. Checkpoint is first in line
Finding where the system failure occurred, the 12:32 event, is relatively quick since only one checkpoint’s worth of information has to be scanned. Once the 12:32 event is located, the dataset is read backwards; all pre-images are reapplied to the files they came from and ROLL BACK processing is complete when the beginning of the dataset is reached.
Reviewing the Role of CCAJRNL/CCARF in RESTART Recovery
CCAJRNL/CCARF records can be used for analysis by auditors, performance analysts, applications managers, DBAs, and system managers. CCAJRNL is also the required input to the REGENERATE facility that restores files damaged by logic or other errors unrelated to Online failures. However, the primary purpose of CCARF is to provide input to ROLL FORWARD processing. But before ROLL FORWARD processing can begin, the checkpoint rolled back to in ROLL BACK processing must be located in CCARF. Quick access to that checkpoint is crucial to getting the system back up and available for production use. V5R1 provides near instantaneous access to that checkpoint, regardless of the size of CCARF, through the use of the BSAM NOTE/POINT facility.
Working with CCAJRNL CCAJRNL continually increases in size because it contains a complete record of the entire run. CCAJRNL contains blocks of many sizes depending on how frequently the currently active journal buffer is written to CCAJRNL. The minimum block size is 6749.
Figure 4. Typical, production CCAJRNL
Figure 4 represents a CCAJRNL for an Online that came up at 8:00 AM. It continues to increase in size during the run and it is not uncommon for the dataset to consume multiple 3390-DASD volumes by the time the Online terminates. Each u represents an atomic update that can be reapplied to a record or field in a particular file as needed.
If a system crash, power failure or other event causes the Online to terminate abnormally, say at 12:32, then CCAJRNL will contain a record of all atomic updates, to all files, that have occurred from the time the run came up until 12:32. However, for the purposes of ROLL FORWARD processing, only the updates that occurred since the last checkpoint are of interest:
Figure 5. The state of CCAJRNL at the time of a failure
After ROLL BACK processing completes and all files have been rolled back to the checkpoint taken at 12:25, the RESTART dataset is closed and the CCARF (CCAJRNL in the run which crashed) dataset is opened. Prior to V5R1, the dataset was read, sequentially, block-by-block from the beginning looking for the 12:25 checkpoint record. That could involve reading tens or hundreds of thousands of blocks, consuming a considerable amount of time, delaying recovery and increasing the Online down time.
Significant Improvement in CCAJRNL/CCARF Processing in V5R1
When you upgrade to V5R1, RESTART recovery is faster by design. In V5R1, to reduce recovery time and increase system availability, the BSAM NOTE/POINT feature was implemented for locating the desired checkpoint record in CCARF.
During the original run, the NOTE/POINT feature captures the physical location on disk of every checkpoint record written to CCAJRNL and saves that location information in the corresponding checkpoint record written to CHKPOINT. If recovery is required, ROLL BACK processing retrieves this location information from the checkpoint record rolled back to in RESTART DD and passes that information to ROLL FORWARD processing. Now, when ROLL FORWARD processing opens CCARF, it uses that location to position immediately to the desired checkpoint record, saving 5, 10, 20, or more minutes of recovery time.
In Figure 6, when the 12:25 checkpoint is written to CCAJRNL during the original run, the physical location on disk is saved as, for example, CC (cylinder)=28, HH (head)=12, R (relative record)=5 in the corresponding 12:25-checkpoint record written to CHKPOINT. Now, all of the data at the beginning of CCARF is skipped, no I/O is required to skip that data and ROLL FORWARD processing will start immediately at the 12:25 checkpoint found at location CC: 28, HH: 12, R: 5 in the CCARF dataset.
Figure 6. Recording the location of a checkpoint and passing it on
Finding the checkpoint in a VSE journal is accomplished as it was prior to V5R1 as IBM restricts the NOTE/POINT facility for sequential data files under VSE.
If your journal dataset is typically quite large, and recovery becomes necessary late in the day after a large amount of data was written to CCAJRNL, you should notice a substantial reduction in the elapsed time required to run ROLL FORWARD processing. This performance enhancement is automatic and requires no new parameters or changes to JCL.
Note: Please make sure that you have applied the maintenance from AUTOFIX EW3025 to ensure that Fast RESTART recovery operations are supported and functioning. If you have not ordered this latest maintenance you may do so at:
http://www.cca-int.com/forms/secure/m204/autofix.htm
Coming Attractions
In the next article of this three part series I will discuss the changes made to enhance the flexibility of the recovery process.
Model 204 Version 5.1 Library Refresh
In August 2003 we recompiled the Model 204 Version 5.1 library and replaced individual files on the CCA web site. The following books were upgraded:
At no cost to you, you can replace the compiled library with the new compiled library and/or download individual files. Go to the CCA web site. Select Customer Support, then select Documentation. On the Documentation page you can choose to:
If you share the Model 204 Library on a server at your site, CCA recommends that you replace the entire compiled library to take advantage of the Acrobat Search feature. PDF files must be compiled together for the index to work properly
Copyright © 2008 Computer Corporation of America. All right reserved. Published in the United States of America.
