File Handling

jBASE can handle data in a variety of forms. There is a built-in feature to access hashed data files:

• Distributed files

• Resilient files

• Directories

• Sequential files

Data held in other forms can be accessed with an appropriate driver, regardless of where the data is stored, the access mechanisms are same.

The CLEAR-FILE command allows you to clear all records from the dictionary file or data section files.

The command syntax is:

CLEAR-FILE {DICT|DATA} filename{,section}

Where,

• filename is the name of the file to be cleared. The file type must be one of the supported jBASE file types. If the file type supports separate dictionary and data files, then the DICT or DATA keywords should be used to clear either the dictionary file or the data file. DATA is assumed by default.

• section is the name of the multiple data section to be cleared.

Examples

CLEAR-FILE File1

or

CLEAR-FILE DATA File1

Clears the data portion of File1.

CLEAR-FILE DICT File1

Clears the dictionary portion of File1 (File1]D).

The COPY command copies specific or selected records from a specified file to the terminal, printer or another file.

The command syntax is:

COPY {DICT} filename{,section} {recordlist} {(options}
PROMPT
TO: {({DICT} filename{,section}} {targetrecordlist}

Where,

• filename is the name of a valid file. The file type must be one of the supported jBASE file types. The DICT keyword can be used to specify that the record or records should be copied from or to a dictionary file.

• section is the name of the multiple data section, if specified.

• recordlist is the list of records (keys) to be copied. If recordlist is omitted, then the active SELECT list is used, if present.

• targetrecordlist is a list of record keys to copy the records to - effectively renaming the copied records. Each key in the targetrecordlist is applied in sequence to the copied records. If targetrecordlist is not specified or contains less keys than there are copied records, the original record keys will be used.

• Options are:

Notes

If you enter <RETURN> at the TO: prompt, the records will be copied to the terminal screen or spooler, depending on the options chosen. In this case, the D option will have no effect.

Example

COPY File1 Record1 (T

Copies Record1 from File1 to the terminal.

COPY File1 Record1 (OD
TO:(DICT File2 Record2

Copies Record1 from File1 to dictionary file File2]D overwriting Record2. Once copied, the original Record1 is deleted from File1.

The CREATE-FILE command allows you to create a new file to use it within the jBASE system. The command allows creation of file type that can be interpreted by the jEDI libraries (unless the concept of a file is redundant to the target system).

The command syntax is:

CREATE-FILE {DICT|DATA} filename{,section} {HASHMETHOD=nnn} {TYPE=tname} {PERM=nnn} {LOG=YES|NO} {TRANS=YES|NO} {BACKUP=YES|NO} {NumBuckets{, BucketMult{, SecSize}}} {NumBuckets{, BucketMult{, SecSize}}}

Where,

The DELETE command deletes specific or selected records from a specified file.

The command syntax is:

DELETE {DICT} filename{,section} {recordlist}

Where,

• filename – The name of a valid file. The file type must be one of the supported jBASE file types.

• DICT – Delete records from the dictionary file.

• section – Name of the multiple data section if specified.

• recordlist – The list of record keys to be deleted. If the recordlist is omitted, then the active SELECT list will be used (if present).

Example

DELETE File1 Record1

Deletes record Record1 from the file File1.

GET-LIST DeleteList
DELETE File1

Deletes all records from File1 that match the record keys selected by the active SELECT list.

The DELETE-FILE command allows you to delete complete file sets, the dictionary, or the data section of a file.

The command syntax is:

DELETE-FILE {DICT|DATA} filename{,section}

Where,

• filename – Name of the file to be deleted. The file type must be one of the supported jBASE file types.

• DICT or DATA – Keywords used to delete either the dictionary file or the data file.

• section – Name of the data section to be deleted.

Notes

The DELETE-FILE command detects inconsistencies in its use and issues suitable error messages.

Beware of creating a file and then immediately deleting it using the DELETE-FILE command. The DELETE-FILE command will respect the JEDIFILEPATH variable and if it finds a file of the same name in a directory earlier in the path, then the current working directory will delete that file. Therefore, it is best to define the JEDIFILEPATH variable as '.' (the current working directory).

Example

DELETE-FILE File1

Deletes the complete file set of File1, comprising the dictionary file, default data section and any multiple data sections.

DELETE-FILE File1,Section

Deletes the multiple data file named Section1 from File1.

DELETE-FILE DATA File1

Deletes the default data file named File1.

DELETE-FILE DICT File1

Deletes the dictionary file of File1 (File1]D).

The jbackup utility provides fast on-line backup facilities and can also be used to check file integrity.

The command syntax is:

jbackup -Option {Inputlist}

Where, inputlist is a file containing a list of files (default stdin) and option can be:

Notes

This jbackup command sets the JEDIFILEPATH environment variable to “.” to prevent the back up of incorrect files.

Use jchmod to change the characteristics of a file. For example,

jchmod -B filename

This allows jbackup to skip 'filename'. Other options can be +B, -O and +O.

jbackup creates a file named jbk*PID as a work file when executed. Therefore, jbackup must be run from a directory, which has write privileges. If the file system or directory is not write enabled, then you will receive the error message ERROR! Cannot open temporary file jbk*PID.tmp.

Examples

For UNIX:

find /home -print | jbackup -P

Reads all records, files and directories under the /home directory provided by the find selection and displays each file or directory name as it is encountered. This option can be used to verify the integrity of the selected files and directories.

jbackup FILELIST -f /dev/rmt/floppy -m1 -v

Reads all files and directories listed in the UNIX file FILELIST and writes the formatted data blocks to the floppy disk device, displaying each file or directory name as it is encountered. The jbackup utility prompts for the next disk if the amount of data produced exceeds the specified media size of 1 Mbyte.

jbackup -Ajbase -S/usr/jbc/tmp/jbase_stats >/dev/null
LIST /usr/jbc/tmp/jbase_stats USING /usr/jbc/jbackup NAME TOTAL SIZE ID-SUPP

Reads all files and directories in home directory of user-id "jbase". Generates statistics information and outputs blocks to stdout, which is redirected to /dev/null. The statistics information is then listed using the jbackup dictionary definitions to calculate the file space used.

For Windows:

jfind C:\users\vanessa -print | jbackup -P

Reads all records, files and directories under the C:\users\vanessa directory provided by the jfind selection and displays each file or directory name as it is encountered. The -P option specifies that the files are not actually backup (print and scan only). It is useful to verify the integrity of the selected files and directories. This command should be run with jshelltype sh rather than jsh.

jfind D:\data -print | jbackup -f C:\temp\save20030325 -m10000 -S C:\temp\stats -v

The jfind command outputs the names of all the files and directories under the D:\data directory. This output is passed to the jbackup command causing it to backup every file that jfind locates, rather than save to tape. This jbackup command creates a backup file C:\temp\save20030325.

Note that jbackup creates the save2003025 file, but the directory c:\temp must exist before running the command. The -m10000 option specifies that the maximum amount of data to back up is 10,000MB (or 10GB), rather than the default 100MB.

The -S option causes file statistics to be written to the hashed file stats. This file should exist and be empty prior to commencing the backup.

The -v option causes the name of each file to be displayed as it is backed up. Due to the pipe character used to direct the output of jfind to jbackup, this command should be run with jshelltype sh rather than jsh.

The jrestore utility provides fast on-line restores from the saves produced by the jbackup utility. It is possible to control jrestore to restore from any file type during the backup from single records to multiple directories. The jrestore utility can also be used to verify jbackup saves.

The command syntax is:

jrestore -Options

Where Options are:

Notes

When using jrestore ensure that you use the standard shell not jsh, otherwise the double quotes and other meta characters will lose their meaning.

In Windows, when specifying path names the backslash must be escaped with a backslash, otherwise the backslash character is removed. For example, C:\\MyApp\\new

Examples

jrestore -f /dev/rmt/ctape -P

Reads formatted files and directories from a streaming cartridge device, displaying each file or directory as it is encountered. This option can be used to verify the tape does not contain any parity or formatting errors and so can be restored at a later date.

jrestore -f /dev/rmt/floppy -v

Reads and restores formatted files and directories from a floppy disk device, displaying each file or directory as it is encountered.

jbackup -Ajbase | jrestore -c"/home/old /home/new"

Reads formatted files and directories from stdin, which is being supplied by jbackup. It modifies all occurrences of path string /home/old to /home/new and then restores files and directories using modified path string.

jrestore -f BACKUP -d".*PAYROLL$"

Reads formatted files and directories from UNIX file BACKUP. It limits restore to any directories whose path name ends in PAYROLL.

jrestore -f BACKUP -h"/CUSTOMERS$" -i".*SMITH.*"

Reads formatted files and directories from UNIX file BACKUP. It limits restore to any hash files whose path name ends in CUSTOMERS, and only restores record IDs containing the string SMITH.

The jchmod command enables you to modify jBASE-specific file attributes such as the resize parameters of a hashed file.

The command syntax is:

jchmod {+options} {-options} file{ file...}

Where,

• +options are:

  • +B – Flags the file to be saved by the jbackup utility.

  • +L – Flags the file to be logged to the record or transaction log.

  • +T – Flags updates to be included as part of transaction boundaries.

  • +R{parms} – Sets the resize parameters. The syntax is the same as for the CREATE-FILE command and includes spaces. The spaces should be retained by quoting the string from the shell.

• -options are:    

  • -B – Flags the file not to be saved by the jbackup utility.

  • -L – Flags the file not to be logged to the record or transaction log.

  • -T – Flags updates not to be included as part of transaction boundaries.

  • -R – Removes any resize parameters from the file.

  • -t – Tabulates and displays statistics of the files. Note this option is exclusive and will cause all other options to be ignored.

• file{ file…} – A list of all the ‘real’ file names to be processed by the command.

Notes

jchmod will ignore files that are not jBASE supported files, or where options do not apply to certain file types. This allows the command to be issued against an entire directory without determining which files are valid for the command. A warning message is issued for any ignored files.

The ‘real’ file name should be specified to jchmod for dictionary and data sections of a file. The DICT file and file,section formats are not supported by this command.

The jgrep utility enables pattern matching of records in one or more jBASE files.

The command syntax is:

jgrep {options} searchstring file {recordlist...}

Where,

• searchstring is the string to search in the files. If the string contains spaces, surround it with quotes or use the -I option described below.

• file is the name of any valid file.

• recordlist is a list of record keys.

• options can be:

Notes

Options may be specified after the file or recordlist by preceding the options with a left parenthesis. For example,

jgrep searchstring file recordlist (options

The -S option enables all records in a hashed file, or all files in a UNIX directory to be searched.

Example

jgrep "ABC DE" SALES CUST0001

Searches the record CUST0001 in file SALES for the string ABC DE.

find . -name "JAC*" -print | xargs jgrep -NS fast

Standard UNIX commands can be used to provide arguments to the jgrep command.

jgrep -ILSN .

Prompts for search strings and then searches the records in all files in the current directory, and all files in any subdirectories. Standard UNIX commands can be used to provide arguments to the jgrep command. Does not pause at the end of each page of output.

The jrchmod utility can be used to change the deallocation strategy of a JR file.

The command syntax is:

jrchmod {options} filename

Where, filename is the JR file to be changed and options can be one or more of the following:

Notes

The default behaviour of JR files is never to deallocate frames on the assumption that they will be reused.

The jrf utility is used to resize one or more files.

The command syntax is:

jrf {options} {filename{,section}}

Where,

• filename is the base name of the file to be resized. The file type must be one of the supported jBASE hash file types.

• section is the name of the data section to be resized.

• options can be one or more of the following:

Notes

Badly sized files can cause severe performance issues. Often over time, internal file size becomes too small for the number of records they contain.

The jrf utility resizes files listed on the command line, selected via an active SELECT list or all hash files in the current directory. By default, the jrf command resizes all valid hash files in the current directory.

This command should only be used on files that are not in use by any application.

jrf and Resilient Files

Resilient (JR) files do not require resizing, but when data is freshly imported, the order within the internal structure is physically unordered. A single resize of an unordered JR file to a JR file orders the structure such that a SELECT on the file will be a sequential disk read from beginning to end, improving performance. This will not affect the speed of direct reads and writes.

JR files by default takes default internal parameters of INTMODS=3,7,19 and EXTMODS=31. A JR resize retains whatever parameters the original file has used. If there is a particularly unusual hashing pattern, then jrf can be forced into a ‘sizing’ data scan with the –F option where various parameters will be tested to attempt to achieve the best data capacity. The –F option would normally only be used on fairly static files.

The jrscan utility is used to verify or display the internal structure of Resilient files.

The command syntax is:

jrscan {options} filename

Where,

• filename is the base name of the JR file to be scanned..

• options can be one or more of the following:

Notes

This is generally used for diagnostic purposes.

The jstat command analyses a hashed file to provide statistics detail on the distribution of records and use of the data space.

The command syntax is:

jstat {options} {filename{,section}}

Where,

• filename is the base name of the file to be analysed. The file type must be one of the supported jBASE hash file types.

• section is the name of the data section to be analysed.

• options are as follows:

The -f option displays additional information pertaining to the free space within the file. The information helps to identify how fragmented the file has become.

As records are added to the file, buckets are allocated for secondary space or to extend large groups. As records are deleted, empty buckets are added to the free space chain associated with the file. When new buckets are required to extend data space for new records, the buckets available on the free space chain will be used wherever possible. This fills up any ‘holes’ that may have been created in the file. If the file is very dynamic, then this process can create fragmentation of the free space and therefore the file. If this becomes excessive, then the file can become very large in relation to the data it contains.

If there are large number of buckets in the free space chain pertaining to the number of buckets allocated to the file, then this may indicate that the file has become fragmented. Use the jrf utility to remap the file and remove fragmentation.

Two figures regarding free space are given in the output (see notes). The total buckets used for free space chain shows how many buckets have been used within the file to record the free space chain. If this is large, then it is a good indication that the file has become fragmented. The total unused (freed) buckets shows how many buckets have been created for extra storage space and then give back to the free space chain. If this is large in relation to the number of buckets in the file, then this is another indication of file fragmentation.

Bucket Information

The r option displays additional information regarding the efficiency of each bucket in the file. The output is a table showing the number of bytes allocated, and the number of bytes used, for each group in the file. This can then be used to judge the distribution of the data within file. You can also extract this information from the output stream using an awk script (or similar) and then perform a statistical analysis of the figures.

Additional Information

The v or verbose option displays the following additional information:

When the v option is combined with the r option, the key and size of each record in the buckets is also displayed.

Machine Readable Information

The m option displays all information (excluding the -r option) in machine readable format and thus allows resizing and analysing programs to extract information in a simple way. This option excludes all other options, except the -d option, which allows the field delimiter to be changed to a character other than <TAB> (0x09).

Information from the m option is presented in the following order:

Notes

The jstat command has the following output:

File filename
 Type = HASH1 , Hash method = 2 , Created Tue Jul 14 02:58 1992
 Buckets = 97 , BucketSize = 512 , SecondaryBucketSize = 512
 Record Count = 5 , Record Bytes = 832
 Bytes/Record = 166 , Bytes/Bucket = 8
 Primary file space: Total Buckets = 97 , Total Bytes = 1996
 Secondary file space: Total Buckets = 0 , Total Bytes = 0

In the above output, most of the fields are self-explanatory. Below are the fields that are not self-explanatory:

• SecondaryBucketSize is the secondary bucket size of a hashed file as calculated at file creation time by the CREATE-FILE command. It specifies the record size beyond which space will be allocated outside the hash bucket itself. The hash bucket will instead contain a pointer to the record.

• Primary file space is the primary file space allocation. This shows how the file relates to the originally allocated number of buckets and allows the effectiveness of the file size. The total number of buckets should match or should be close to the number originally allocated to the file.

When a record is to be stored in a bucket that does not have enough free space, the bucket is allocated an additional number of primary buckets to accommodate the new record. This can be done in two ways:

1. If you have allocated enough buckets to cater the number of records in the file, then increasing the size of each individual bucket should allow the records to fit into the file. The new bucket size should be large enough to cater for the average record size shown in the jstat output.

2. If there are many records than the number of buckets, then you should increase the number of buckets to reflect this. The new number of buckets should be equal to the number of records divided by the number of average size records that will fit into a single bucket.

If the file is growing in size over time, you should allow some additional expansion in your calculations. You may find that a combination of both the above techniques is necessary if the average record size is much greater than the current bucket size and there are many more records than allocated buckets.

If the records in your file are very large, it is usually more efficient to force all records into secondary file space rather than create a very large bucket size. In this case, recreate the file with a secondary bucket size of 0.

Secondary file space indicates how much of the file was allocated as pointers to secondary space in the original group. Where possible, this should only form a small proportion of the file.

In general, the file should be sized slightly larger than its ‘perfect’ size. Unless files are very badly sized, overall performance will only be affected marginally.

The SEL-RESTORE command restores all or specific records into a jBASE file from an ACCOUNT-SAVE or FILE-SAVE.

The command syntax is:

SEL-RESTORE targetfilename {recordlist} {(options}

Prompt

Name of Account on media: sourceaccountname
Name of File on media: sourcefilename

or

Number of File on media: sourcefilenumber

Syntax Elements

options are as follows:

Notes

The SEL-RESTORE command prompts for the name of source account and file held on the media, unless the N option (restore by file number) has been used (in which case you will be prompted for the number of the source file on the media).

Before execution, any tape device should have been opened with the T-ATT command.

Examples

SEL-RESTORE NEWCODES * (I
Name of Account on media: PAYROLL
Name of File on media: TAXCODES

Restores all records from the TAXCODES file in PAYROLL account into the NEWCODES file. The record key is displayed as each record is restored.

SEL-RESTORE NEWCODES SINGLE (N
Number of File on media :22

Restores record SINGLE from file number 22 into the NEWCODES file.

A Distributed file is a collection of existing files used primarily for organizing data into functional groups. Each file within the collection is called a part file. A distributed file can contain up to 254 part files. The method for determining in which part file a record belongs is called the partition algorithm.

For example, your database consists of records, which span 42 regions and you elect to distribute your data so that each part file contains all records for a specific region. With distributed files you would be able to process any one of the region part files independently of the others, or you would be able to process all 42 region part files collectively (as one database containing the records from all 42 regions).

Distributed files can also be used when the size of a file exceeds the size limit for the operating system (typically 2 gigabytes). This effectively permits file sizes to reach 254 times the maximum file size your operating system allows.

The components of a distributed file collection are called part files. These files can have any name and can be any file type except a distributed file.

Part files can exist anywhere on the network, accessible via the JEDIFILEPATH environment variable, Q-pointers or F-pointers. Each part file is assigned a part number when it is attached to a distributed file. The part number must be a positive integer in the range of 1 through 254 inclusive. This part number is an integral element as it is used by the partition algorithm to determine which part file the record belongs.

Part numbers do not have to be sequential nor do they have to be continuous. For example, 4 part files numbered 52, 66, 149 and 242.

A part file can belong to more than one distributed file although this imposes two restrictions:

• The part file must always have the same part number for each distributed file to which it belongs.

• All distributed files to which a part file belongs must use the same partition logic. In other words, when a record is written to the common part file, the partition algorithm for each distributed file must resolve the record's location in the same manner. This is only applicable when the distributed file uses the user-defined partition method.

The number of part files and the partition algorithm can be varied at any time throughout the life of the distributed file. Be aware that if the partition algorithm changes such that records that were normally written to one part file, using the original partition algorithm, might to be written to another part file using the new partition algorithm. This could lead to unwanted duplication.

Another issue that can occur is the wrong file can be accessed through the distributed file stub (the file to which the part files are attached to create the distributed file set; see Creating Distributed Files). Be aware that part files are resolved in the same manner as any other file in jBASE.

For example, suppose two files exist with the same filename where one is resolved through an F-pointer (in $JEDFILENAME_MD) and another is resolved via $JEDIFILEPATH, and that the one in $JEDIFILEPATH is our actual part file. What will happen is the actual part file will never be found because the file pointed to by the F-pointer will be found first, as indicated by the jshow -f command. To alleviate this issue, it is best to attach the files using a full explicit file path.

A distributed file is created using the CREATE-FILE command with the qualifier TYPE=DISTRIB. This creates two files, a dictionary which is a Hash4 (currently fixed at mod3) and the distributed file stub. If desired, the dictionary can be resized using the jrf utility. For example, the following command creates a distributed file called DISTREGION:

jsh DEMO ~ -->CREATE-FILE DISTREGION TYPE=DISTRIB
[ 417 ] File DISTREGION]D created , type = J4
[ 417 ] File DISTREGION created , type = DISTRIB

The file partition table is empty at this point, and the partition algorithm is set to the default system partition method with a delimiter of ‘-‘ (that is; all record IDs must be of the form "PartNumber-recordID"). These aspects of the distributed file can be changed with the create-distrib command.

Files are attached to a distributed file using the create-distrib command with the -a option. A file must already exist before it can be attached to a distributed file.

In the following example, an existing file DISTCUST.SOUTH is attached to the distributed file DISTCUST as part number 4:

jsh DEMO ~ --> create-distrib -a DISTCUST 4 DISTCUST.SOUTH
Part file 'DISTCUST.SOUTH', Part number 4 added

This method of attaching a distributed file is preferred to ensure the proper part file is resolved through the partition algorithm. See Part Files for more details.

A part file can be detached from a distributed file using the create-distrib command with the -d option. The synonym DELETE-DISTRIB can also be used for this purpose. For example, to detach the DISTCUST.SOUTH part file:

    jsh DEMO ~ --> create-distrib -d DISTCUST 4
    Part file 'DISTCUST.SOUTH', Part number 4 deleted

Each distributed file uses a partition algorithm to determine in which part file a record belongs. The partition algorithm is specified by using the create-distrib command. All part files belonging to a distributed file uses the same partition algorithm.

There are two methods for defining the partition algorithm, the system defined method and the user-defined method. The partition algorithm uses the record ID (or part of the record ID) to distribute the record to the appropriate part file.

System Partition Algorithm

When a distributed file is created, it is automatically set to use the system partition algorithm. The create-distrib command is used to set the partition algorithm. When using the system partition method, jBASE assumes that the record ID is in the form:

    <PartNumber> <Delimiter> <RecordID>

Where,

• PartNumber is an integer, which determines the part file to which the record is written.

• Delimiter can be any character except a system delimiter (AM, VM, SVM). The default delimiter is a dash (-).

• RecordID is the actual item-ID of the record. In a ‘hashed’ file type, this determines the group to which the record is written.

The following example sets (or changes) the distributed file DISTREGION to use the system partition algorithm. A dash (-) is used as the delimiter between the part number and the record ID:

    create-distrib -pSYSTEM,- DISTREGION

User-defined Partition Algorithm

The user-defined method provides the greatest flexibility when the need to determine which part file a record belongs is not based on an integer value (although it could be used for that purpose as well). The user-defined method is implemented by writing a jBC subroutine, which determines the part number, and then setting the distributed file to use the subroutine as the partition algorithm. As an example, a distributed file called DISTCUST has 5 part files and the first character of each ID determines which part file the record belongs.

The following jBC subroutine is used as the partition algorithm:

SUBROUTINE DistCustSub (Reserved, Key, PartNo)
EQU Otherwise TO 1
* Extract the first character from the Key
FirstChar = Key[1,1]
* Now determine the PartNo based on FirstChar
BEGIN CASE
CASE FirstChar = "N"; * North
    PartNo = 1
CASE FirstChar = "E"; * East
    PartNo = 2
CASE FirstChar = "W"; * West
    PartNo = 3
CASE FirstChar = "S"; * South
    PartNo = 4
CASE Otherwise ; * Invalid condition; isolate for later review
    PartNo = 99
END CASE
*
RETURN

Compile and catalog the subroutine. Ensure that the subroutine is accessible via the JBCOBJECTLIST environment variable.

The subroutine is called each time a record is read from or written to the DISTCUST distributed file. The subroutine must support the following arguments:

Argument	Description
Reserved	This parameter is reserved for future enhancements and should not be altered within the context of the subroutine.
Key	This is the record ID. It must be constructed in the application program prior to READing or WRITEing the record from or to the distributed file. Do not alter this argument; use it only as a source.
PartNo	This must be assigned by the subroutine and must return a valid part number.

You will notice that the part numbers consist of 1, 2, 3, 4 and 99. This illustrates an important feature. It is not a requirement that the part numbers be sequential or continuous. This could be used to allow additional part files to be added to the distributed file collection without the necessity of renumbering.

Take special care when writing this subroutine to account for all possibilities. If for any reason the PartNo cannot be determined, you will receive either a READ_ERROR or WRITE_ERROR at the point of failure. Here is one such example where there are 11 part files. The part number is determined based on the last character of the key, the last character is assumed numeric but, if it is not, then it will be placed in the 11th part file:

SUBROUTINE distsub(reserved, key, partno)
lastchar = key[-1,1]
IF NUM(lastchar) THEN
    partno = lastchar
    IF partno = 0 THEN partno = 10
END ELSE
    partno = 11
END
RETURN

Can you spot the error?

The fatal flaw is if the subroutine ever encounters an item-id of 'null'. A null item-id is considered numeric, hence partno will be set to 'null'. A better way to code this would be:

SUBROUTINE distsub(reserved, key, partno)
lastchar = key[-1,1]
IF lastchar >= 0 AND lastchar <= 9 THEN
    partno = lastchar
    IF partno = 0 THEN partno = 10
END ELSE
    partno = 11
END
RETURN

This subroutine takes the 'explicit' approach and does not make assumptions about what form the data will be in.

To set (or change) the distributed file to use the user-defined partition algorithm, use the create-distrib command. For example, to set the DISTCUST distributed file, use the DistCustSub subroutine:

    create-distrib -pUSER,DistCustSub DISTCUST

When compared to the system partition algorithm, the user-defined partition method incurs a small performance penalty when calling the jBC subroutine. The exact cost of this is highly dependent on how easily the part number is resolved within the subroutine.

The CREATE-DISTRIB command is used to maintain the parts of a distributed file, set or change the partition algorithm and list or verify the component part files and the partition algorithm.

The CREATE-DISTRIB command accepts a variety of options, which determines its function as the following table illustrates:

CREATE-DISTRIB -a FileName PartNo PartFile

CREATE-DISTRIB FileName PartNo PartFile

CREATE-DISTRIB -d FileName PartNo

DELETE-DISTRIB Filename PartNo

CREATE-DISTRIB -l FileName

LIST-DISTRIB FileName

CREATE-DISTRIB -v FileName

VERIFY-DISTRIB FileName

CREATE-DISTRIB -pSYSTEM{,Delim} FileName

CREATE-DISTRIB -pUSER,Subroutine FileName

The table below describes the syntax elements:

The other options that can be used are:

• -f – Force Mode

• -V– Verbose

Examples

CREATE-DISTRIB -a INVOICES 24 INVOICES.MAR1999

Attaches the file INVOICES.MAR1999 as the 24th part file to the INVOICES distributed file.

CREATE-DISTRIB INVOICES 24 INVOICES.MAR1999

Same as the previous example. Note that the -a option is assumed in the absence of any options.

CREATE-DISTRIB -pSYSTEM,| DISTCUST

Sets (or changes) the partition method to the SYSTEM partition algorithm for the DISTCUST distributed file. A vertical bar "|" is used as the delimiter to separate the part number from the record ID.

CREATE-DISTRIB -pUSER,DistSub DISTCUST

Sets (or changes) the DISTCUST distributed file to use the user-defined subroutine DistSub as the partition algorithm.

CREATE-DISTRIB -d MEDICAL.CLAIMS 149

Detaches (disassociates) the 149th part file from the MEDICAL.CLAIMS distributed file.

CREATE-DISTRIB -l CONVENTIONS

Lists the component part files of the CONVENTIONS distributed file. Also lists the partition algorithm.

CREATE-DISTRIB -v CONVENTIONS

Verifies the existence of the component part files belonging to the CONVENTIONS distributed file. Also confirms the partition method. If the distributed file uses the user-defined partition method, then this also verifies that the subroutine can be executed.

The DELETE-DISTRIB command detaches (de-references) a component part file from a distributed file. The command syntax is:

DELETE-DISTRIB FileName PartNumber

Where,

• FileName – The name of the Distributed File.

• PartNumber – An integer from 1 through 254 inclusive which was used to associate the Part File to the Distributed File.

Notes

If the user-defined partition method is used, then you should ensure that the subroutine used for the partition algorithm does access the de-referenced file.

If the system partition method is used, then you should ensure that no keys are created which can read from or write to the de-referenced file.

Example

DELETE-DISTRIB INVENTORY 42

Detaches (de-references) the 42nd part file from the distributed file INVENTORY.

The LIST-DISTRIB command displays all partition information pertaining to a distributed file. The command syntax is:

LIST-DISTRIB FileName

Where, FileName is the name of a Distributed File.

Notes

The VERIFY-DISTRIB command is much more useful as this not only displays the same information as LIST-DISTRIB, it also verifies the existence of the component part files.

Example

LIST-DISTRIB INVENTORY

The VERIFY-DISTRIB command verifies the existence of the component part files of a distributed file. If the distributed file uses the user-defined partition method, then VERIFY-DISTRIB also verifies that the subroutine is executable. The command syntax is:

VERIFY-DISTRIB FileName

Where, FileName is the name of a Distributed File.

Example

VERIFY-DISTRIB INVENTORY

Although jBASE does not restrict you from directly populating part files, records should always be written through the distributed file stub. Be aware that if a record is placed in the wrong part file, and that record is subsequently handled through the  partition algorithm, it will be placed in the part file according to the partition algorithms own relentless logic. This will result in the same record appearing in two part files.

Once part files are populated, changing the logic of the partition algorithm (or changing the partition method), could have disastrous results. If it is necessary to do this, you must pass each record through the new partition algorithm so that it is placed in the proper part file. You must also remember to delete each record from its original location.

A distributed file is opened in the usual way. For example, the following statement opens a distributed file called DISTCUST:

    OPEN "DISTCUST" TO DISTCUST_FILE ELSE ABORT 201,"DISTCUST"

By default, when a distributed file is opened, all component part files are opened at the same time. You can defer the opening of all part files by setting the JEDI_DISTRIB_DEFOPEN environment variable.

On versions of jBASE prior to 3.3.9, if a record ID resolved to a partition (part file) that did not exist, the process would be trapped to the jBASE debugger with an "Error 22" error message. This behavior has been changed (see patch number PN3_30268) such that a READ from a non-existent partition will take the ELSE clause and a WRITE will be trapped with an 'Error 22' unless the WRITE is supplied with the ON ERROR clause.

If you delete a part file then you must also use DELETE-DISTRIB to remove the reference from the distributed file stub. You must also modify any user-defined partitioning algorithm. This is detailed in the distributed file example.

Distributed files support secondary indexes and triggers at both the distributed file level and the part file level.

In this comprehensive example, you create a distributed file called DISTCUST using a user defined partition algorithm and attach five part files.

1. Create the distributed file stub. This is the file to which all part files will be attached.

   jsh DEMO ~ -->CREATE-FILE DISTCUST TYPE=DISTRIB
   [ 417 ] File DISTCUST]D created , type = J4
   [ 417 ] File DISTCUST created , type = DISTRIB
   

2. Define the partition algorithm. If the distributed file uses the default system partition method, then this step would not be necessary unless you wanted to change the delimiter separating the part number from the record ID. For this example, you will use the user-defined method by assigning the subroutine DistCustSub as the partition algorithm.

   jsh DEMO ~ --> create-distrib –pUSER,DistCustSub DISTCUST

3. Create the five files to be attached as part files. If the files already exist, then you can omit this step.

   jsh DEMO ~ -->CREATE-FILE DISTCUST.NORTH 1 101
    [ 417 ] File DISTCUST.NORTH]D created , type = J4
    [ 417 ] File DISTCUST.NORTH created , type = J4
   jsh DEMO ~ -->CREATE-FILE DISTCUST.EAST 1 101
    [ 417 ] File DISTCUST.EAST]D created , type = J4
    [ 417 ] File DISTCUST.EAST created , type = J4
   jsh DEMO ~ -->CREATE-FILE DISTCUST.WEST 1 101
    [ 417 ] File DISTCUST.WEST]D created , type = J4
    [ 417 ] File DISTCUST.WEST created , type = J4
   jsh DEMO ~ -->CREATE-FILE DISTCUST.SOUTH 1 101
    [ 417 ] File DISTCUST.SOUTH]D created , type = J4
    [ 417 ] File DISTCUST.SOUTH created , type = J4
   jsh DEMO ~ -->CREATE-FILE DISTCUST.ERRORS 1 41
    [ 417 ] File DISTCUST.ERRORS]D created , type = J4
    [ 417 ] File DISTCUST.ERRORS created , type = J4
   

4. Attach the five files to the distributed file.

   jsh DEMO ~ -->create-distrib -a DISTCUST 1 DISTCUST.NORTH
    Part file 'DISTCUST.NORTH', Part number 1 added
   jsh DEMO ~ -->create-distrib -a DISTCUST 2 DISTCUST.EAST
    Part file 'DISTCUST.EAST', Part number 2 added
   jsh DEMO ~ -->create-distrib -a DISTCUST 3 DISTCUST.WEST
    Part file 'DISTCUST.WEST', Part number 3 added
   jsh DEMO ~ -->create-distrib -a DISTCUST 4 DISTCUST.SOUTH
    Part file 'DISTCUST.SOUTH', Part number 4 added
   jsh DEMO ~ -->create-distrib -a DISTCUST 99 DISTCUST.ERRORS
    Part file 'DISTCUST.ERRORS', Part number 99 added
   

5. List the part files you have just added.

   jsh DEMO ~ -->LIST-DISTRIB DISTCUST -or- create-distrib -l DISTCUST
   Partitioning Algorithm is USER Subroutine 'DistCustSub'
   Part file 'DISTCUST.NORTH', part number 1
   Part file 'DISTCUST.EAST', part number 2
   Part file 'DISTCUST.WEST', part number 3
   Part file 'DISTCUST.SOUTH', part number 4
   Part file 'DISTCUST.ERRORS',part number 99
   

   The distributed file system is now complete. At this point, the DistCustSub subroutine is called each time a record is written to or read from the DISTCUST file.

   For example, you find that the DISTCUST.ERRORS part file is not required. You should take the following actions to remove this file from the distributed file:

• Detach the part file from the distributed file.

  DELETE-DISTRIB DISTCUST 99

  or

  create-distrib -d DISTCUST 99

• Modify the DistCustSub user-defined partition subroutine by removing the lines, which allocate records to part number 99. Recompile and catalog.

• Optional, delete the DISTCUST.ERRORS file.

  DELETE-FILE DISTCUST.ERRORS