Computer Corporation of America
|
Feedback
Search CCA:
   
USA CCA
CCA Products
CCA Customer Support
CCA Resources
CCA - Company
CCAPRINT: A Newsletter for Model 204® and System 1032® Users
April 24, 2008
     
Model 204: Large Objects and Table E in V6R1.0 Printer-friendly version
System 1032: Library Maintenance Printer-friendly version

Model 204
USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.

Large Objects and Table E in V6R1.0
By James Damon



Hundreds, if not thousands, of different types of data objects can now be created and manipulated by an ever-increasing number of software packages and hardware devices. Files with extensions representing text documents, photographs, spreadsheets, videos, slide shows, and sounds --the list seems endless--have proliferated in every environment. Figure 1 shows the icons associated with a few of the applications used to process these objects.

 

.doc .wav .xls gmail .pdf .jpg
Figure1. Application icons and file extensions

Many Model 204 customers wanted to be able to store, manage, locate and retrieve some of these data objects using a Model 204 file. This would provide the ability to link a data object to text data: for example, a picture of a house as a .jpg file linked to a street address. Responding to this need, in V6R1.0 CCA implemented support for these data objects, calling them large objects. Now, any data object, regardless of length or type of data (binary, character or a mixture) can be stored, retrieved and updated in a Model 204 file. Suport for large objects is provided through the following:

  1. New file parameters and commands
  2. New User Language statements
  3. A new table, Table E, in a Model 204 file

When stored in a Model 204 file, large objects, with the exception of unformatted xml and text documents, generally contain binary data that is in a format unknown to Model 204. A Model 204 file is primarily a storage and retrieval facility for these large objects. Editing, displaying and manipulating large objects is still handled by well-known, third-party software packages that are readily available.

Table E Architecture
When a file is created under V6R1.0 and the new parameter, ESIZE, is greater than or equal to 20, Table E will be created and initialized for use as a large object repository.

CREATE FILE LOBFILE
PARAMETER ESIZE=50000
END CREATE
OPEN LOBFILE
INITIALIZE

Viewing all of the Table E parameters—ESIZE, EHIGHPG, EPGSUSED--immediately after the file is initialized, but before any data has been loaded, would show the following:

VIEW ESIZE,EHIGHPG,EPGSUSED
ESIZE 50000 PAGES IN TABLE E
EHIGHPG 16 TABLE E HIGHEST PAGE USED
EPGSUSED 17 TABLE E PAGES IN USE

The first 17 pages of Table E, numbered zero through 16, are reserved for the segment bit map and free page bit maps. Additional bit map pages are allocated throughout Table E when required.

Each page in Table E is formatted as part of initialization. The page trailer is the last 40 bytes of each page and the final four bytes contains the table and page number of that page. The 1-byte table number for Table E is X’40’. The six remaining bits in this byte as well as the remaining three bytes on the page are available for constructing the page number. This means that the maximum number of pages in Table E is X’3FFFFFFF’, or 1,073,741,823 pages.

Large Objects
A large object occupies contiguous pages, as many as required, in Table E. There is no limit, except for ESIZE, to the number of pages a large object may consume. For example, a 30 second .avi video clip, in low resolution, can consume about six megabytes or about 975 Table E pages. Pictures taken with a digital camera can be anywhere from 32K to 3M in length, depending on resolution, or five to 480 Table E pages.

Figure 2 is a schematic of what the first twelve large object pages might contain in Table E.

Figure 2. Table E large objects

X’40000011’

X’40000012’

X’40000013’

X’40000014’

X’40000015’

X’40000016’

X’40000017’

X’40000018’

X’40000019’

X’4000001A’

X’4000001B’

X’4000001C’

These pages could represent twelve different large objects or each collection of icons could represent the pages required to store that single, large object. This collection of Table E pages shows that the first page available is page number X’11’ = 17 which is the 18th page in the file. At this point, a VIEW command would show the following:


VIEW ESIZE,EHIGHPG,EPGSUSED
ESIZE 50000 PAGES IN TABLE E
EHIGHPG 28 TABLE E HIGHEST PAGE USED
EPGSUSED 29 TABLE E PAGES IN USE

Enlarging Table E
If additional pages are required, you can issue an INCREASE TABLEE command. Like INCREASE TABLEB and INCREASE TABLED, the INCREASE TABLEE command will allocate pages in FREESIZE to Table E. If there are no pages in FREESIZE, the INCREASE DATASETS command can be used to add additional datasets to the file. This will increase the number of pages in FREESIZE making those pages available for the INCREASE command.

In Summary
If you have an application in mind that could combine text data with graphics, spreadsheets, pictures, maps, and so on, then large objects provide the solution.

Coming attractions
In part 2 of this article, I’ll review the facilities available for storing, locating, retrieving, updating, printing and deleting large objects as well as the tools available for reorganizing files containing large objects.

 

 

System 1032
USE OF AND ACCESS TO PRODUCTS AND FEATURES ARE IN ACCORDANCE WITH THE TERMS AND CONDITIONS OF THE USER’S SOFTWARE LICENSE. THE PRESENTATION OF MATERIAL HEREIN DOES NOT, IN ANY MANNER, MODIFY SUCH TERMS AND CONDITIONS.

System 1032 Library Maintenance
By Tym Stegner

TymIn today’s System 1032 environments, many people maintain System 1032 libraries according to processes passed along by previous employees. A basic understanding of what a library is and how it works is assumed, but, in fact may be missing. When in-house protocol fails them, customers call concerning issues of maintenance for library catalogs. These questions range from the basic--"How do I get my procedures into this library?"--to the complex--"I got a fatal error when I inserted this procedure into my library."

This article will cover the structure of a System 1032 library and what you can do to avert library problems.

Looking at Your Libraries
According to The System 1032 User’s Guide, a library catalog is a repository for procedures and other System 1032 tools—including System 1032 Forms, Record Descriptors (RDs), and variables—that you use when working with data in your databases and datasets. When you store tools in a library catalog, they are stored in what is called intermediate code. Thus, the tool code runs faster than if the tool code was compiled each time it was run.

By issuing a series of SHOW commands, you can display and examine what is stored in a particular library. Generally, the full source code for a tool is available in its library. In Figure 1 we used a FORMGEN-created library of the DIRECTORS dataset. (For more information about using the System 1032-supplied FORMGEN utility, see System 1032: Forms Feature, Part 1 in the November 2005 issue of CCAPRINT.)

Figure1. Various SHOW commands displaying library elements

1032> show lib directors_lib
Library DIRECTORS_LIB
1032> show lib directors_lib procedure
Library DIRECTORS_LIB
Procedure MOVE_DIRECTORS_REC
Procedure ADD_DIRECTORS_REC
Procedure CHANGE_DIRECTORS_REC
Procedure FIND_DIRECTORS
Procedure CORRECT_DIRECTORS
Procedure ADD_DIRECTORS
Procedure UPDATE_DIRECTORS
Procedure DISPLAY_DIRECTORS
Procedure SELECT_DIRECTORS
Procedure MAIN_DIRECTORS
1032> show lib directors_lib form
Library DIRECTORS_LIB
Form DIRECTORS_FORM
Form DIRECTORS_MENU
1032> show lib directors_lib rd
No RD found in Library DIRECTORS_LIB
Library DIRECTORS_LIB
1032> show lib directors_lib variable
Library DIRECTORS_LIB
Variable DIRECTORS_CRITERIA Group of
DIRECTOR_ID Integer
FIRST_NAME Text 15
SURNAME Text 15
YEAR_OF_BIRTH Date_Time
COUNTRY_OF_BIRTH Text 15
CITY_OF_BIRTH Text 15
End_Group
1032>

The full definition of a library can be obtained using the DEFINITION quality of the SHOW LIBRARY command. A much smaller library is used for the Figure 2 example.

Figure2. Examining the MYLIB definition

1032> show lib MYLIB definition
Library MYLIB;
Variable TEXT_ONE Text Varying;
Form MYFORM
Field TEXTER Text 100 Prompt "Enter data"
End_Form;
Variable TEXT_TWO Text 355;
Procedure MYPROC;
Variable myNow Date_Time
Let myNow = $now
Write myNow format("Hello, world, it’s " d3.3)
End_Procedure;
End_Library;
1032>

Library Structures
Structurally, a library catalog can be considered an ordered list of objects organized by type of element. Each tool element is presented according to its order of definition, so that later elements can refer back to previously defined elements. Elements in the list may contain child elements, which will immediately follow their parent elements. For example, in Figure 2 the Form MYFORM element is followed by a child element: Field TEXTER Text 100 Prompt "Enter data". Any elements that can be referenced by another catalog are included in the list. This includes any attribute references in procedures defined within the library.

For example, in Figure 3 a simple library list might read as follows:

Figure 3. From a simple library, MYLIB


Variable TEXT_ONE Text Varying
Form MYFORM
Form_Field TEXTER Text 100 (child of MYFORM)
Variable TEXT_TWO Text 355
Procedure MYPROC;
Variable myNow Date_Time (local to MyPROC)

However, imagine a library created by FORMGEN or one of your own applications. The FORMGEN created library DIRECTORS_LIB, for example, has the following object element breakdown:

The previous list does not enumerate all the form-field, global, and local variable references, which are also significant to the ordered list.

Compiled References
All library elements are positionally dependant within the library and type dependant as well, because the reference to the element assumes a particular type of element and additionally for variables or form-fields, a data type. This information is particularly important when other libraries refer back to your library. The concept of positionally related dependencies between libraries and datasets are referred to as compiled references. (For more about compiled references, please see: System 1032 User’s Guide, Module 4, page 65, "Using System 1032 Compiled References.")

Updating a Library
It is customary at some sites that elements stored in a library are updated using REMOVE and INSERT commands. When a new element is inserted into a library, the reference structure must be updated to include all the new element references that were added at the end of the list.

However, when an element is removed from the list, it is not merely a matter of deleting the references to that element. The positions of all the subsequent elements must still be preserved, because those positions are significant to other libraries. In other words, compiled references to the remaining elements must be preserved. (The details of this process are beyond the scope of this article.)

Therefore, over the course of many REMOVE and INSERT cycles, the internal structures that track the components of the internal reference list become increasingly fragmented. Eventually, this can lead to a situation where an insert into the library fails.

To resolve errors that occur during INSERT or REMOVE processing is usually easy: just recreate the library. This action completely refreshes the internal reference list. The problems are gone until the next time the reference list becomes too fragmented again.

Note that the REPLACE command functionally does a REMOVE followed by an INSERT. However, INSERT and REMOVE processing allows you to add elements to a library and REPLACE does not. You can use REPLACE commands to make changes to existing code, or the non-structural elements of variables, form-fields, and RD fields.

Refreshing a Library by Recreating It
Library catalogs are formed in a fashion similar to that of the other System 1032 catalogs, by using the CREATE command. Use the DESCRIPTION option to read the commands from a DMD file or use interactive mode to define or import tools. In either case, first you must be sure to open any other catalogs that are referenced by the library you want to compile, as shown in Figure 4.

Figure 4. Creating the DIRECTORS_LIB Library

$ s1032
Computer Corporation of America System 1032 Version V9.82-0
Copyright 2007, Computer Corporation of America
1032>
1032> open dataset DIRECTORS in s1032_demo readonly
Current dataset is now DIRECTORS
1032>
1032> open lib FORMGEN3 in s1032_tools readonly
1032>
1032> create description DIRECTORS_LIB.DMD output []
1032>
1032> show library *
Library FORMGEN3
Library DIRECTORS_LIB
1032>

The FORMGEN3 library contains global variables and procedures used by all FORMGEN-created libraries.

Note in Figure 4 that the more recently created library, the DIRECTORS_LIB library is listed following the previously opened library(s).

You can also populate a library using the INSERT command. After creating a simple library as shown in Figure 5, the INSERT command is used thereafter to add the remaining elements to the library.

Figure 5. Populating a library via INSERT commands


$ s1032
Computer Corporation of America System 1032 Version V9.82-0
Copyright 2007, Computer Corporation of America
1032>
1032> create library MYLIB
1032 Data Definition Language
DDL> variable text_one text varying
DDL_ form myform
DDL_ FORM statement: field texter text 100 prompt "Enter data"
DDL_ FORM statement: end_form
DDL_ end_library
1032>
1032> insert in library MYLIB
DDL> variable text_two text 355
DDL_ end_insert
1032>
1032> insert in library mylib description myproc.dmc
1032>
1032> show lib MYLIB definition
Library MYLIB;
Variable TEXT_ONE Text Varying;
Form MYFORM
Field TEXTER Text 100 Prompt "Enter data"
End_Form;
Variable TEXT_TWO Text 355;
Procedure MYPROC;
Variable rightnow date_time
Let rightnow = $now
write rightnow format("Hello, world, it’s " d3.3)
end_procedure;
End_Library;
1032>

Summary
This article has reviewed the structural layout of the System 1032 library catalog, and looked briefly at several commands intrinsic to the maintenance and use of libraries, including SHOW, CREATE, REPLACE, INSERT and REMOVE. An understanding of the background of library catalogs and how they are manipulated is useful to all System 1032 programmers and maintainers.

Copyright © 2008 Computer Corporation of America.
All right reserved. Published in the United States of America.


Contact CCA Webmaster
Copyright 2008