Extracting Metadata from Datasets, Part I By Tym Stegner For some applications, it is often useful to acquire attribute or dataset contextual information programmatically. The classic examples of this are determining if an attribute is keyed or extracting attribute formats to generate a report. This two-part article explores how to obtain attribute and dataset information and suggests a method for storing user-defined dataset or attribute context data for retrieval. In this Part I we address the SHOW command API.
Using the SHOW Command The basic means of obtaining attribute or catalog—dataset, database or library--environmental or contextual information is the SHOW command. SHOW command output displays elements for any of the primary classes within System 1032. The list of classes includes:
To obtain metadata from a SHOW command programmatically for use in data or context manipulation, we must send SHOW command output to an external file--usually it is displayed at the terminal. To redirect the output of a SHOW command to an external file use the following syntax:
SHOW [ ON { filespec or # } ] class…
You can specify either a new file specification or the channel number of a previously initialized output channel.
Now you could issue a SHOW ON filespec command for a piece of metadata, then open and read in the external file and parse out the required element. However, a more direct methodology exists.
Using the SHOW Command API Perhaps you have used the Host Language Interface version of the SHOW command, the DM_SHOW procedure. DM_SHOW is an API that provides 3GL access to the System 1032 SHOW command. The DM_SHOW API uses an OPENVMS feature that can return any datatype passed via VMS descriptors. However, DM_SHOW is available only at the 3GL level.
As the System 1032 tools procedures do not support VMS descriptors, individual data-type dependant procedures are required at the PL1032 level. Fortunately, a set of tools procedures provides the necessary functionality at the PL1032 level. These tools are the SHOW_datatype tools procedures. The following is a partial list of the tools procedures relevant to metadata extraction:
Extracting Metadata In the following example, the FORMAT quality of the TOTAL_RENTALS attribute is required. The appropriate dataset and the tools library containing the SHOW_datatype procedures are opened.
1032> Open Ds FILMS In S1032_DEMO ReadonlyCurrent dataset is now FILMS1032> Open Library S1032_HLI in S1032_TOOLS Readonly1032>1032> Show Atr TOTAL_RENTALS FormatAttribute TOTAL_RENTALS Decimal Keyed Format (N$,8.2)1032>
Once the required catalogs are opened, a SHOW command is issued that reminds the programmer of the input and output parameters required by the SHOW_TEXT tools procedure:
1032> Show Proc Show_Text ParamProcedure SHOW_TEXT BUFFER Output Text QUALITY Text Varying TEXT_IDENT Text Varying Optional INT_IDENT Integer Optional CLASS Text Varying Optional1032>
A single text output is required, so a variable is defined. In declaring the parameters:
1032> Var Fx Text Varying1032> Call Show_Text(Fx,"FORMAT","TOTAL_RENTALS",,"ATTRIBUTE")1032> Print Fx FX--------------------(N$,8.2)1032>
Once the appropriate parameters are set, the procedure call is made, and the result is returned in the defined variable. Note that in this example the parentheses are part of the numeric format, not value delimiters. The parentheses ensure that negative values are enclosed in parentheses.
In Summary In this part of the “Extracting Metadata from Datasets”, we covered the use of the SHOW command API via the SHOW_datatype tools procedures to extract contextual metadata from a given dataset. In the concluding article, we will look at how to access metadata not available to the SHOW command API, as well as suggest a method for storing your own metadata in a dataset.
Footnote 1: Tables of class qualifiers are listed on pages 426-429 of the System 1032 Programmer’s Reference Manual. The individual classes and qualities listed in those tables are described in detail on pages 429-434.
Model 204 USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.
Reorganizing Files Containing Large Objects in V6R1.0, Part IV By James Damon
In CCAPRINT July 2008 I highlighted the User Language statements that support the retrieving, updating, and deleting of large objects in a Model 204 file. In this final article on large objects, I’ll discuss the User Language statements and file management facilities that support the reorganization of such files.
Unloading Large Objects Ahe easiest way to unload a file for reorganization is by using the User Language PRINT ALL INFORMATION (PAI) statement. This also holds true for large objects. Several options for the PAI statement that were introduced in V6R1.0 support previewing large objects. These options specify how and what to print for records containing large objects. All other fields are printed as usual fieldname=value pairs. The options for large objects include:
Large Object PAI options
Prints…
PAI LOB_FLOD
Output in a binary format that FLOD or FILELOAD can read to load a Large Object, which includes the Large Object itself and the Large Object descriptor
The following output was produced by issuing each of these PAI statements against one record containing the large object, fieldname=SUMMARY. This large object field is the first 450+ bytes of the Wikipedia summary of “Moby Dick” that we used in last month’s CCAPRINT article. OUTCCC was also set to 51 (column 51 is reserved for the continuation character). The following procedure is the application that produced the output:
BEGINFR WHERE AUTHOR = MELVILLE PAI LOB_FLOD NEW PAGE PAI LOB_DATA NEW PAGE PAI LOB_SHORT_DATA NEW PAGE PAI LOB_NO_DATA NEW PAGE PAI LOB_NONEEND
PAI LOB_FLOD Statement Output Use the PAI LOB_FLOD (or just PAI) statement, when you are ready to unload the data for file reorganization. This statement produces output that includes the large object descriptor in binary format, required for FLOD or FILELOAD.
SUMMARY = ?? ?z??{ ? | Aiming to join a whaling crew, Ishmael heads for N-antucket, the older of the two U.S. centers of the- whaling industry. Time problems force him to sto-p for the night in the new, more powerful whaling -center of New Bedford, Massachusetts. Lacking mon-ey, he lodges at the Souter Inn. The innkeeper, P-eter Coffin, puts him in a room with the mysteriou-s tattooed cannibal, Queequeg, a harpooner. Despi-te Ishmael's initial reservations, the two become -friends.AUTHOR = MELVILLESOURCE = WIKIPEDIA
PAI LOB_DATA Statement Output Use the PAI LOB_DATA statement on a small sample of records to examine a few large objects and to gauge the size of your large objects prior to the full unload with PAI LOB_FLOD.
SUMMARY = (length 700) (reserve 0) Aiming to join a whaling crew, Ishmael heads for N-antucket, the older of the two U.S. centers of the- whaling industry. Time problems force him to sto-p for the night in the new, more powerful whaling -center of New Bedford, Massachusetts. Lacking mon-ey, he lodges at the Souter Inn. The innkeeper, P-eter Coffin, puts him in a room with the mysteriou-s tattooed cannibal, Queequeg, a harpooner. Despi-te Ishmael's initial reservations, the two become -friends.AUTHOR = MELVILLESOURCE = WIKIPEDIA
PAI LOB_SHORT_DATA Statement Output Use the PAI LOB_SHORT_DATA statement to display the total length of large objects but only the first 255 bytes of data.
SUMMARY = (length 700) (reserve 0)Aiming to join a whaling crew, Ishmael heads for N-antucket, the older of the two U.S. centers of the- whaling industry. Time problems force him to sto-p for the night in the new, more powerful whaling -center of New Bedford, Massachusetts. Lacking mon-ey, h AUTHOR = MELVILLESOURCE = WIKIPEDIA
PAI LOB_NO_DATA Statement Output The PAI LOB_NO_DATA statement is useful for showing only the length of a large object without displaying the data.
SUMMARY = (length 700) (reserve 0)AUTHOR = MELVILLESOURCE = WIKIPEDIAA
PAI LOB_NONE Statement Output Use the PAI LOB_NONE statement if you want to unload the file without large objects or if you just want to preview a few records minus their large objects.
AUTHOR = MELVILLESOURCE = WIKIPEDIA
File Reorganization To reorganize the file using the current data in the file you must first unload the data. You may use PAI or your own, fixed format unload. If you decide to use your own unload for large object fields, you must construct the large object descriptor in binary as described in “Building a Large Object Descriptor” in the Model 204 File Manger’s Guide for V6R1.0.
Assuming you use the PAI unload format, you can also use the standard FLOD or FILELOAD program to reload the data from the standard TAPEI dataset.
Summary Files containing large objects do not require any special handling as far as file reorganization is concerned. The standard PAI to unload the data followed by the standard FLOD or FILELOAD program to reload the data is all that is required. Most organizations already have standard file reorganization jobs, which can be used unchanged, to accomplish this task.
Copyright © 2008 Computer Corporation of America. All right reserved. Published in the United States of America.
