System 1032 USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.
Back-to-Basics Part 7 - Dataset Creation & Dataset Tools By Tym Stegner
In our previous articles in this series, we covered the fundamentals of dataset design, focussing on how to define attributes, attribute features and options. Once you design all the attributes for a dataset, it is time to create that ubiquitous System 1032 data structure: the dataset.
Creating a Dataset Using a CREATE command you can define a System 1032 dataset, which is the basic unit for accessing and manipulating data. Datasets are defined using the dataset definition syntax, and always contain definitions of their component attributes and those attributes options in addition to the data the attributes describe.
A dataset may also optionally contain elements of one or more of the following tools:
Examining Dataset Structure and Ancillary Files Datasets store their metadata internally and a SHOW command reveals the definitions of all the attributes contained within the dataset. Additionally, the SHOW command reveals the definitions of all the included tools within the dataset, with the exception of procedures marked NO_SOURCE.
Introducing NO_SOURCE Procedure Security When you define a procedure with the NO_SOURCE option, a SHOW command cannot display the procedure. This prevents others from seing how your application was actually coded.
Introducing Base and Ancillary Files Attribute data and attribute keys are also stored within the dataset. System 1032 data records are not actually stored as records, but rather in chains of data blocks. To foster access and/or update efficiency, you can define the data storage and/or key storage to external files, which are then placed on alternate disks from one another.
The base file description for a dataset is file-name.DMS, for example: MYFILE.DMS. The same dataset name is commonly used for any ancillary dataset file, but with a differing file type suffix, which is specific to the type of ancillary file. For example, if you decide to:
Damage history records longer than 2K bytes are stored in an external file with file description MYFILE.DMU.
Setting Up a Base Dataset The syntax for the creation of a dataset comprises the CREATE command, as follows:
CREATE DATASET ds-name [DESCRIPTION dmd-name] [OUTPUT location]
You can abbreviate the DATASET keyword to DS and the DESCRIPTION keyword to DESC.
Writing the Definitions to Create a Dataset You can enter a dataset’s definition in one of two methods.
In either mode, you can also copy attribute definitions from any currently opened dataset using the COPY attribute-name command. With this command, you can clone any attribute from any open dataset into the dataset you are currently defining. This is very helpful when creating a series of datasets using some similar attributes.
Note: For quick projects and demonstrations, it is easier to write code interactively, because online help is so readily available. However, until the interactive dataset definition is completed, there is no way to preserve the current attribute definitions should an unexpected erasure of your work occur. To ensure against loss of any work you have entered at the terminal, CCA recommends against interactive code development.
Detailing the OUTPUT Option The primary purpose for the OUTPUT option is to relocate a new dataset to a location other than the current directory.
When a dataset (or other catalog) is created, System 1032 stores within the dataset its current location, its location at time of creation, as well as the location specified in the OUTPUT command. This data can be displayed using the SHOW DS FILE command:
1032> OPEN DS FILMS IN S1032_DEMO READCurrent dataset is now FILMS1032> SHOW DS FILEDataset FILMS File: SYS$SYSDEVICE:[S1032.V9811.DEMO]FILMS.DMS;1 Typed String: S1032_DEMO: Expanded String: DDSK:[DEMO_DVL.SYS]FILMS.DMS1032>
In the previous example the line labeled File shows the current physical location of the dataset FILMS. The line labeled Typed String represents the value supplied to the OUTPUT option; in this case, the logical name S1032_DEMO. The Expanded String label represents the original physical location at the time the dataset was created. When the FILMS dataset was originally created the physical locations entered at the File and Expanded String labels matched.
Understanding the Importance of Logical Names Logical names are commonly used to locate the DMJ, DMK, and DMR files. (The DMU file is automatically created in the same directory location as the base DMS file.)
CCA recommends that you specify the location identified in the OUTPUT option as a logical name. A logical name is also a better choice for naming any dataset that is used in either a database or with compiled procedures. Or, any time you expect to relocate files, for example, from development into production, create logical names to specify the directory location, rather than hard-code the location (which you would then have to find and change for every instance).
Without a logical name specified in the OUTPUT option, future implicit opens done upon the dataset will be working with a physical location for the dataset which may not match where the dataset has been moved for production. If you use a logical name, you must redefine only the logical name(s) to ensure that implicit opens for the dataset work as expected. More information regarding the CREATE command, implicit opens, and compiled references is in the System 1032 User’s Guide, Module IV, Chapter 3.
Customizing Dataset Components As previously described, you can define the records and keys within a dataset to be external to the base DMS file structure. These tasks are defined with the options of the dataset definition clauses: LOAD_DEFAULT and KEY_DEFAULT.
The LOAD_DEFAULTS option controls optional dataset features relating to I/O operations upon the dataset:
The KEY_DEFAULTS option controls dataset features that relate to indexing operations:
Creating Other Catalogs The CREATE command is also used to generate the other types of System 1032 catalogs. You can use:
A System 1032 database can also contain tools similar to those described for libraries, but it is primarily used for sets of datasets that are to be logically joined together for PRINT command or retrieval operations. A database also stores the join relationships among the datasets comprising the database.
In Summary The CREATE command generates a System 1032 dataset containing attribute definitions and other optional components. A dataset stores its own definition for later retrieval via SHOW commands. The basic dataset file uses the DMS file type, and any ancillary dataset-related files have type-specific file types, but share the dataset name. Datasets can be created interactively, or from a prewritten description file. Logical names should be used for any ancillary file locations, as well as in the OUTPUT clause, to make future compiled references to the dataset work correctly. The CREATE command is also used for the other System 1032 catalogs: databases and libraries.
In Conclusion This series of articles describing how to define attributes and create datasets has reached its end point with a description of how to actually create a dataset, while briefly touching upon non-dataset catalog creation. This series has discussed some of the reasoning that goes into how attributes and dataset are defined, as well as exposing some of the possibly lesser-known options and qualities of attribute and dataset definition.
We welcome suggestions for future technical articles regarding the System 1032 enviroment, the use of System 1032, or how to accomplish work using System 1032. Please send any such suggestions to 1032_Support@CCA-INT.COM.
Model 204 USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.
Automatic Increase for BSIZE and DSIZE By James Damon
The number of pages in a Model 204 file can be increased dynamically using the INCREASE TABLEB or INCREASE TABLED commands. If there are insufficient pages in FREESIZE to accommodate the increase, new pages can be added, also dynamically, to FREESIZE using the INCREASE DATASETS command. When data growth can be anticipated, these increases can be made in advance and file availability remains unaffected. However, these increases are not automatic. You must anticipate the increase required and issue the commands manually prior to running update transactions that require the additional pages.
Unexpected transaction volumes or unanticipated data growth can increase requirements for additional pages beyond what is currently allocated to BSIZE or DSIZE. When this occurs, files are marked full, requests are cancelled, and transactions are backed out. The newest release of Model 204, V6R1.0, provides an automatic facility for adding pages to those tables when required and avoids those negative consequences. The new facility automatically reallocates pages from FREESIZE to the appropriate table based on parameters set by the file manager or DBA.
Enabling the Automatic Increase Feature To use the automatic increase feature, files must be created under V6R1.0. This may require that you reorganize existing files. Whether you identify existing files that would be good candidates or create new files, you can set the parameters BAUTOTYP, BAUTOINC, and DAUTOINC in entry order, unordered, and sorted files. You can also set DAUTOINC in hashed files.
Automatically Increasing Table B Two parameters control the feature in Table B: BAUTOINC and BAUTOTYP.
BAUTOINC specifies the number of unused pages to maintain in Table B. The number of unused pages in an entry order, unordered, or sorted file is BSIZE minus (BHIGHPG + 1)*. BAUTOTYP specifies when to perform the automatic increase. If set to: X’00’ No automatic increase occurs in Table B when it becomes full X’01’ Automatically increase the number of pages in Table B when it becomes full X’02’ At each physical open, if the number of unused pages is less than BAUTOINC, add a number of pages (possibly more than required by BAUTOINC) to increase the number of segments in the file by one. This is done only if the number of records in the last segment is greater than one quarter of a segment: 49,152 divided by 4, or 12,288.
When an automatic increase occurs, pages are logically reallocated from FREESIZE to Table B.
* The previous asterisk (*) and subsequent asterisks (*) note where the original edition of this newsletter was incorrect and has been updated to reflect correct information.
In the following example, the parameters listed have these values when the file is physically opened—which means that no other user, including APSY and User 0, currently has the file open.
Note: In the YOURFILE examples the letters A, B, C, D, and E in blocks represent pages in Table A, Table B, and so on. The letter F represents pages in FREESIZE.
Automatic Increase in Table B at File Open The number of unused pages, at the time of the physical open, is:
BSIZE - (BHIGHPG + 1) = 3*
The number of unused pages Model 204 should maintain in the file is:
BAUTOINC=6
Therefore, three* pages are allocated to Table B from FREESIZE as part of the physical open.
When BAUTOTYP=X’02’ or X’03’, an additional calculation is done. If the number of records that currently exist in the last file segment is less than one quarter of a segment (which is 49,152 divided by 4 or 12,288) then nothing else is done. However, if the number of records currently existing in the last file segment is greater than one quarter of a segment, BSIZE is increased by the number of pages necessary to increase the number of potential file segments by one. This may result in an increase in BSIZE greater than BAUTOINC.
This can be done only when the file is physically opened, since at that time requests referencing the file have not yet been compiled. Once requests are compiled that refer to a specific file, the number of segments in that file may not be increased.
Automatic Increase in Table B during Request Evaluation If Table B becomes full during request evaluation and BAUTOTYP=X’01’, then if BAUTOINC pages are available in FREESIZE, Table B is increased automatically by BAUTOINC pages. If BAUTOINC pages are not available, then all available pages in FREESIZE are used. If the number of available pages is sufficient for the current transaction then request cancellation and transaction back-out will be avoided, and this greatly improves file availability.
Automatically Increasing Table D Only one parameter controls the feature in Table D: DAUTOINC.
DAUTOINC specifies the number of unused pages, in addition to reserved pages (DPGSRES), to maintain in Table D. The number of unused pages in Table D is: DSIZE - (DPGSUSED + DPGSRES)
For Table D, the algorithm is much simpler. At physical open, the number of unused Table D pages is computed and, if necessary, DSIZE is increased using pages from FREESIZE to maintain DAUTOINC unused pages in Table D. If insufficient pages are available in FREESIZE, whatever pages are available are used.
During request evaluation, if the current update transaction would cause Table D to fill, then DSIZE is automatically increased by DAUTOINC pages, if available, from FREESIZE. If FREESIZE has fewer pages than DAUTOINC, whatever pages are available are used. If the number of available pages is sufficient for the current transaction then request cancellation and transaction back-out will be avoided.
In the following example, the parameters listed have these values when the file is physically opened. There are no unused pages in Table D, if DPGSRES is used in the calculation. Therefore, Table D will be increased. However, since only five pages are available in FREESIZE, the maximum increase that can occur in DSIZE is five pages.
Employing the INCREASE DATASETS Command When FREESIZE is exhausted, you can add new pages dynamically with the INCREASE DATASETS command. First, allocate a new dataset, perhaps using the ALLOCATE command as follows:
ALLOCATE YOURF2 WITH SCOPE=SYSTEM SEQUENTIAL CATALOG - VOLUME=CCAVOL UNIT=3390 PRI 10 CYL BLKSIZE=6184 - DSN=CCA.YOURFILE.YOURF2.M204
Second, issue the following commands.
OPEN YOURFILEUPDATE PASSWORDINCREASE DATASETS WITH YOURF2
Now you will have added all of the pages in YOURF2 to FREESIZE in YOURFILE.
YOURFILE
The logical Model 204 file named YOURFILE is now composed of two datasets and two DDNAMES: YOURFILE and YOURF2. You can issue an INCREASE DATASETS command as many times as necessary to add new datasets and new pages to FREESIZE.
In Summary The automatic increase feature for Table B and Table D represents a significant step along the way to 24/7 file availability. As you begin to implement this feature in your critical production files, request cancellations and transaction back-outs should be reduced, at least for the TABLE FULL event, and this should result in greater file and subsystem availability.
