Computer Corporation of America
|
Feedback
Search CCA:
   
USA CCA
Rocket
Customer Support
CCA Company
CCAPRINT: A Newsletter for Model 204® and System 1032® Users
February 10, 2006
     
Model 204: Dynamically Rebuilding the Reuse Queue with the BLDREUSE Command Printer-friendly version
System 1032: Questions and Answers Printer-friendly version

 

Model 204
USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.

Dynamically Rebuilding the Reuse Queue with the BLDREUSE Command
By James Damon



As part of our ongoing commitment to increase system availability, CCA implemented the BLDREUSE command in V6R1.0. This command provides for the dynamic, online rebuilding of the reuse queue in a Model 204 file that was defined as unordered, reuse record numbers (RRN). You can run the command even though the file is opened for update by many users. You do not have to stop application subsystems or bump users who have the file opened. However, before I describe the command, I need to explain the reuse queue.

The Reuse Queue
When a record is deleted with the DELETE RECORD statement in a FOR EACH RECORD loop, the Table B space consumed by the record is reclaimed and all related index entries are updated. This is not the case for the DELETE RECORDS IN statement (a logical delete) but that’s a different topic. Reclaimed Table B space is always reusable. However, the record number made available by a DELETE RECORD statement is not reusable unless the file has been created as an RRN file. This is achieved at file CREATE by setting the file parameter FILEORG=X’24’.

An RRN file is unordered because new records may use the record numbers of previously deleted records. Therefore, the order of records in an unordered file is unpredictable; the order is neither chronological nor entry order, but instead unordered. The advantage of this file type is that in a volatile file with many record deletions, the storing of new records does not necessarily increase the physical size of Table B. The RRN feature helps to minimize the number of file segments, increases Table B utilization and at the same time reduces the risk of filling Table B.

Introducing the Reuse Queue
The RRN feature is implemented with the construction of a queue of Table B pages known as the reuse queue. These pages are linked together by a pointer within each page pointing to the next page in the chain. Each page, when initially placed on the queue, must have at least one free record number and contain the minimum amount of free space (expressed as a percentage of 6144 bytes) required by the BREUSE file parameter. Figure 1 shows the schematic of the reuse queue.

Figure 1. Reuse Queue schematic

You can display the number of pages in this reuse queue by issuing the VIEW BQLEN command, as shown in Figure 2.

Figure 2. VIEW BQLEN output

VIEW BQLEN
BQLEN 6 TABLE B QUEUE LENGTH

Reuse Queue in Operation

DELETE RECORD Processing
At the completion of a DELETE RECORD statement, if the Table B page where the record used to reside contains at least BREUSE amount of free space, then that page is added to the tail of the reuse queue through the following sequence of events:

  1. The next-page pointer on the previous Tail of Queue page is updated to point to the new tail page
  2. The next-page pointer on the new tail page is set to minus one
  3. The Tail of Queue pointer in the FPL is updated to point to the new tail page.

STORE RECORD Processing
During a STORE RECORD operation against an RRN file, BHIGHPG is the first page used to store new records.

M204.1230: TABLE B FULL -- APPENDS --: filename

Maintaining the Reuse Queue
Over time, a page on the reuse queue may no longer be eligible to be on the queue, because of changes in free space on the page. This can happen as records expand and consume more space on the page. However, pages are removed from the reuse queue only when a STORE RECORD statement attempts to use the page and fails due to insufficient space.

Other pages that are eligible may not be on the queue for the same reason. A record was deleted from a page but at the time the page had insufficient free space to be eligible for the queue. But, over time, free space has become available due to field deletion or field contraction. Pages are added to the reuse queue only during DELETE RECORD statements so many eligible pages in this state may not be known.

Changes to file parameters such as BRESERVE or BREUSE may also change the eligibility requirements for adding pages to or removing pages from the reuse queue. Prior to V6R1.0, file reorganization was the only way to rebuild the reuse queue.

Introducing the BLDREUSE Command
The BLDREUSE command is available in V6R1.0 for dynamically rebuilding the reuse queue during normal system operation. You no longer have to stop subsystems or bump users. In other words, system availability is not affected by rebuilding the reuse queue in files where it is required. The syntax of the BLDREUSE command is illustrated in Figure 3.

Figure 3. BLDREUSE command syntax

BLDREUSE {NEW | [FROM nn][TO nn]}
or
IN filename BLDREUSE {NEW | [FROM nn][TO nn]}

The NEW keyword indicates that the reuse queue is to be completely rebuilt by scanning every Table B page to determine its eligibility for the queue. Use of the NEW keyword requires exclusive access to the file and can only be used when no other users have the file open.

The FROM and TO keywords do not require exclusive file access. If a BLDREUSE command is issued without keywords, a FROM 0 TO BHIGHPG clause is the default. Otherwise, only Table B pages in the range specified are examined for eligibility. Eligible pages are added to the queue when the FROM and TO keywords are used. (However, ineligible pages are not removed.)

Figure 4 illustrates two results of the BLDREUSE command after resetting the BREUSE parameter.

Figure 4. BLDREUSE output examples

VIEW BREUSE
BREUSE 20 FREE SPACE REQUIRED TO REUSE TABLE B PAGE
RESET BREUSE 10
BREUSE 10 FREE SPACE REQUIRED TO REUSE TABLE B PAGE
BLDREUSE NEW
TABLE B QUEUE LENGTH BEFORE REBUILD: 33
NUMBER OF PAGES THAT WERE ON QUEUE: 33
TABLE B QUEUE LENGTH AFTER REBUILD: 7857
=================================================================
RESET BREUSE 5
BREUSE 5 FREE SPACE REQUIRED TO REUSE TABLE B PAGE
BLDREUSE NEW
TABLE B QUEUE LENGTH BEFORE REBUILD: 7857
NUMBER OF PAGES THAT WERE ON QUEUE: 7857
TABLE B QUEUE LENGTH AFTER REBUILD: 25053

In Summary
If you think a file could be approaching a Table B full condition and many records are being stored and deleted, first view BQLEN. If the number is small or you think that there are more pages which should be eligible for the reuse queue, issue the BLDREUSE command. You could experiment by first using the FROM and TO keywords against, say one tenth or one quarter of the file. If the results indicate that significant numbers of pages were added to the queue, it may be worthwhile to use the NEW keyword and rebuild the entire queue. You may also want to reduce the value of the BREUSE file parameter to ensure that more pages are eligible for placement on the queue. Note that the command places an additional burden on the disk buffer pool, so running it during off hours is probably a prudent approach.

 

System 1032
USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.

Questions and Answers
By Tym Stegner

TymCan I check which version of a dataset I have opened by looking at its file specification from within a procedure?

Had this been a question about an interactive session, the answer is easy: a SHOW DATASET FILE command, which displays the embedded file data for the dataset, stored at the time of dataset creation, as follows:

1032> OPEN DS FILMS IN S1032_DEMO READONLY
1032> SHOW DATASET FILE
Dataset FILMS
File: SYS$SYSDEVICE:[S1032.V9811.DEMO]FILMS.DMS;4
Typed String: S1032_DEMO:
Expanded String: DDSK:[DEMO_DVL.SYS]FILMS.DMS

Where in the output:

However, the question asks to check programmatically. Again we turn to the SHOW command, but instead we use the callable interface to the SHOW command, available from the S1032_HLI tools library, the SHOW_TEXT procedure. Unfortunately, FILE is not an available option for the SHOW_TEXT procedure.

So, we will use the SHOW command, but output the information to memory, then parse out the portion we are specifically interested in: the current file specification.

To accomplish this, we will use two tools procedures, found in the S1032_HLI library, available in the S1032_TOOLS file area. The procedures are INIT_OUTPUT and GET_OUTPUT.

The INIT_OUTPUT procedure lets you output to a memory channel, and GET_OUTPUT lets you retrieve data from a memory channel into a variable. The sequence of commands necessary to extract the currently open dataset’s file specification into a variable is:

1032> SHOW DS FILE
Dataset FILMS
File: SYS$SYSDEVICE:[S1032.V9811.DEMO]FILMS.DMS;4
Typed String: S1032_DEMO:
Expanded String: DDSK:[DEMO_DVL.SYS]FILMS.DMS
1032> OPEN LIB S1032_HLI IN S1032_TOOLS READONLY
1032> SHOW PROCEDURE INIT_OUTPUT PARAMETER
Procedure INIT_OUTPUT
CHANNEL Integer
1032> SHOW PROCEDURE GET_OUTPUT PARAMETER
Procedure GET_OUTPUT
OUTBUF Output Text
OUTPUT_LINES Output Integer
CHANNEL Integer
NUM_LINES Integer Optional
1032>
1032> VARIABLE flspec TEXT VARYING
1032> VARIABLE nmlines INTEGER
1032>
1032> CALL INIT_OUTPUT(8)
Initializing channel 8
1032> SHOW ON CHANNEL 8 DATASET FILE
1032> CALL GET_OUTPUT(flspec,nmlines,8,2)
1032>
1032> WRITE FLSPEC
Dataset FILMS
File: SYS$SYSDEVICE:[S1032.V9811.DEMO]FILMS.DMS;4
1032>
1032> LET flspec = flspec[$FIND("File:",flspec,1,1)+6:$LEN(flspec)]
1032> WRITE flspec
SYS$SYSDEVICE:[S1032.V9811.DEMO]FILMS.DMS;4
1032> RELEASE CHANNEL 8
1032>

Begin by doing a SHOW PARAMETER of the procedures to check what must be passed into the procedures. For the INIT_OUTPUT procedure, we must supply only the channel number, which does not need to be a variable. I chose channel 8. In contrast, calls to the GET_OUTPUT procedure do require us to define variables, as data is passed back, so a text buffer and an integer buffer are declared.

To perform the work, first initialize a memory channel. Then, use the SHOW command to write the file information into the memory buffer. Next, a call to the GET_OUTPUT procedure directs the procedure to read the first two records from channel 8, and return the text data into the text variable. Although it is not displayed, the nmlines variable contains 2.

Writing out the contents of the text buffer variable shows the first two lines of the SHOW FILE output. Since we want only the dataset file specification in the variable, use the System 1032 substring operator ("[n:m]") to extract the file specification itself, by positioning to the end of the title string "File: " using the $FIND system function. The m value for the substring operator is the current length of the string.

Once we have the value, release the memory channel in the normal fashion.

I must generate hundreds of data files from a dataset that will later be used to create a new copy of a dataset. What are my options to programmatically create the proper file names from attributes and/or variables? Oh, by the way, the output data is in binary.

Up to the last aside, this is a straightforward request. A facility exists just for this sort of case, the $FILE system function. $FILE lets you pass a text expression to be used as a file specification.

1032> Variable Filename Text Varying
1032> Let Filename = $Text($Now,D5.5),-
Filename = $Repall(":","",Filename),-
Filename = $Repall(".","",Filename),-
Filename = $Repall(" ","",Filename)
1032> Initialize Channel 8 $File(Filename&".DMI")
Initializing channel 8
1032> Write Filename
0602101451510241190

$FILE, however, works only with the INITIALIZE command. As the question involves binary data, the DUMP command is necessary, and the $FILE system function does not work with the DUMP command.

Historically the solution is to build the file name into the command via text concatenation. This is done either in HLI via the DM_EXEC (or DM_CMD) statements, or via use of the EXECUTE command. For example, in FORTRAN, the file name is part of the text expression passed.

Filnam = ‘FIN95’

Stat = DM_EXEC(‘DUMP DATA ‘//Filnam//’.DMI’)

In a similar fashion, you can use the EXECUTE command within System 1032 to build command expressions to run. The problem with using EXECUTE processing is that the command operates one command level above where it is defined. If you use it within a procedure, any variables necessary to create the command expression must be declared outside of the procedure itself, so that EXECUTE processing can reference them for execution.

Fortunately, a third method has recently presented itself. In this solution, we make use of a VMS technology: logical names. By using the SET_LOGICAL tools procedure, we can define a temporary logical name to represent the file specification we want to dump data into. The DUMP command specifies the logical name as the DATA_OUTPUT parameter value. If the logical name is redefined before the next use of the DUMP command, the output goes into a separately named file.

Variable FLX Text Varying
Variable CTX Integer
Open Dataset FIN99
Repeat
Call Find_file(FLX,CTX,"*.DMI")
If $Cvtmiss(FLX,"X") Ne "X" Then
Call Set_logical("DMIFILE",FLX,)
Append Data DMIFILE: Rd Rgx_rd
Find All
Call Rename_file(FLX,"*.DMX")
End_if
Until $Cvtmiss(FLX,"X") Eq "X"
Call Find_file_end(CTX)

The previous example reads DMI files from the current directory, and appends each sequentially into the FIN99 dataset. Once a file is appended, it is renamed to allow a skip to the next file. Note the DMIFILE logical being set to the input file name, after which the logical name is used in the APPEND command.

You can use this approach with bulk data commands within HLI programs as well, though you may want to access the logical declaration utility (LIB$SET_LOGICAL) specifically.

 


Contact CCA Webmaster
Copyright 2007