Command list

The following is a list of all Unicage commands and a description of their function.

DATABASE

ajoin1	Match on the longest key field
and	Outputs the field values that two files share in common in the first field
block_getlast	Within each block of records that share the same value in their key field, outputs all of the records at the end of the block that share the same value in the reference field.
ccnt	Counts the columns
cjoin0	Sequential key matching select
cjoin1	Sequential key matching join
cjoin2	Sequential key matching join
count	Count the number of rows that contain the same key
csum	Returns a cumulative sum
delf	Remove the specified field delf=Delete Field
delkey	Deletes the key field from consecutive records.
dmerge	Merges files based on a key
fldcmp	Compares files field-by-field
fsed	Replaces characters in a field
getfirst	Outputs the first row with a matching key.
getlast	Outputs the last row with a matching key.
getno	Return a sequential number
hsum	Horizontal sum.
interlace	Merges records alternating rows
join0	Selects the lines from Master with a matching key field
join1	Joins a master file to a transaction file. (Only rows with matching key fields are selected)
join1x	Joins to files where more than one record has matching key values.
join2	Joins a master file to a transaction file. (For rows that match, joins data from master for rows that do not match, joins dummy data.)
join2x	Joins to files where more than one record has matching key values.
joinx	Perform a complete join with all possible combinations.
keycut	Split a file based on a key field (Key must be sorted)
lineup	Returns the unique data in the specified field in ascending order
loopj	Join all records from multiple files. loopj = loop of join3
loopx	Join by all possible combinations.
msort	In-Memory Sort
order	Returns the order of the record within a block of records that share the same key value.
perlsql	Executes an SQL command
psort	Partial Sort
rank	Add a rank
self	Outputs the data from the specified field self=select field
selr	Selects rows that match exactly
sm2	Sum values by key
sm3	Sum values by key (No Sort)
sm4	Inserts subtotal lines.
sm5	Output the grand total
sorter	Split a file based on a key (The file doesn’t need to be sorted on the key)
tagcjoin0	Outputs tag data that exists in the tag master. Records that don’t exist are discarded.
tagcjoin1	Joins together tag data that exists in the tag master. Records that do not exist are discarded.
tagcjoin2	Joins together tag data that exists in the tag master. Records that do not exist are padded.
tagcount	Count the number of rows that contain the same key
tagdelf	Remove the specified field from a tag file and output the result
taggetfirst	Outputs the first row with a matching key.
taggetlast	Outputs the last row with a matching key.
tagjoin0	Outputs tag data whose tags exist in the Tag Master Records that don’t exist are discarded.
tagjoin1	Outputs tag data whose tags exist in the Tag Master Records that don’t exist are discarded.
tagjoin1x	Joins together tag data that exists in the tag master in all possible combinations. Records that do not exist are discarded.
tagjoin2	Joins and outputs tag data whose tags exist in the Tag Master. Records that don’t exist are joined with dummy data.
tagjoin2x	Joins together tag data that exists in the tag master in all possible combinations. Records that do not exist are padded with dummy data.
tagjoinx	Joins tag data in all possible combinations
taglineup	Returns the unique data in the specified field in ascending order
tagloopj	loopj command for tag files
tagloopx	Join by all possible combinations.
tagmsort	In-Memory Sort for tag files
tagpsort	Partial Sort for tag files
tagself	Outputs data from the specified field of a tag file.
tagsm2	Sum values in a tag file by key
tagsm3	Sum values by key (No Sort)
tagsm4	Inserts subtotal lines.
tagsm5	Output the grand total
tagupl	upl for tag-formatted data
underlay	Overlay fields one record at a time.
up3	Merges two files on the same key field
upl	Merges two files on the same key field and then extracts the final line of a group of lines that share the same key value.
ychange	Creates a record showing changes from the previous record

DATE/TIME

calclock	Calculates date and time (calendar clock)
dayslash	Conversion filter for date/time formatting
isdate	Checks an 8-digit date
mdate	Performs date calculations
time-excel	Convert a Date and/or time to Excel date/time format

FORMATTING

1x	Remove leading zeroes
bb	BashBeautifier A filter that beautifies Bash code. Performs the following: Adds indents Lines up all of the pipes
bedit	Binary Editor
calc	Insert data or formula immediately after the specified field. wrapper for “awk” command
calsed	sed “light”
cap	Converts roman alphabet letters to upper case.
charsplit	Cuts a field into two fields of fixed length
comma	Formats the specified field with commas every 3rd (or 4th) digit.
ctail	Delete the last n rows of a file
distribute	Writes standard output into separate files one line at a time.
drawline	Draws a line
fcols	Fix (align) the columns in a text file
filemrg	Merge a file into a template
formfix	Fixes (makes read only) all forms in an HTML file
formmerge	Merges characters into an HTML template
fromcsv	Filter
hcat	Concatenates files horizontally
head2	Replacement for head command
lcat	Repeated output of file (looping cat)
linecut	Splits a file into the specified number of lines
linefeed	Adds a linefeed to the end of a file.
makec	Converts row data into columns.
maker	Converts columnar data into rows.
map	Converts a list to a matrix with row and column headings
name-tag	Converts a file in name format to tag format
overlay	Overlay one file on top of another.
pad0	Add zeros to the beginning
sb_pipealign	Aligns the pipes in a bash script sb_pipealign=shellscript beautifier for pipe alignment
sb_rtrim	Removes whitespace from the right of lines in a bash script sb_rtrim = shell beautifier for right side triming
strmerge	Merge characters into a template
tagawk	A wrapper for awk that allows tag names to be used
tagcat	cat multiple tag files
tagcomma	Add commas to the specified field every three (or four) digits
tagcond	A wrapper for awk that allows tag names to be used
tagmap	Converts a tag file list to a matrix with row and column headings
tagmerge	Merge characters into a template
tagunmap	Restore a file that was tagmapped to its original format
tdconnect	merges <td></td> blocks
tocsv	Converts a space delimited file to a CSV file
transpose	Transposes rows and columns
trconnect	merges <tr> elements
txtmv	Moves the position of text.
unmap	Converts a mapped file back to its original form.
unsco	Remove leading and trailing underscores from fields
vcat	Concatenates files vertically
wexcel	Merge data into an Excel report template
wexcelx	Merge data into an Excel 2007-2010 report template (xlsx,xlsm)

INPUT/OUTPUT

cgi-name	Converts data received using the CGI POST method to “name” format.
cgi-read	Reads data that was passed using the CGI POST method
cgi-tag	Converts data passed using the CGI POST method to tag format.
check_attr_name	Checks the attributes of data in name format
check_date_name	Validates the date/time data in name format
check_dble_name	Checks for duplicate data in name format
check_inlist_name	Checks data in name format to see if values are on a specified list of values.
check_need_name	Checks if a particular value exists in data in name format
email	SMTP client (mail sending client)
htable	Extracts strings from a table in an HTML file.
mime-name	Converts MIME format data to name format
mime-read	Reads a MIME formatted file
mnameread	Outputs multiple name formatted files (in tag format)
nameread	Read a file that is in “name” format
rexcel	Convert text in an Excel file.
rexcelx	Converts an Excel file to text
wpdf	Create a PDF
xdump	Displays a hexadecimal dump of a file

tion.

MATHEMATICAL

check_cmp_name	Compares data in name format
divk	Divide the specified field by 1000
lcalc	Perform precision floating point calculation (18 integer digits + 18 decimal places)
plus	Adds the arguments
proportion	Returns a proportion
rand	Generates random numbers
ratio	Returns a ratio
round	Round, round-up or truncate
taground	Round, round-up or truncate

STRING

iscode	Checks the number of digits in a numeric code.
isnum	Checks if the argument is a number
kana	Converts Japanese “Katakana” to/from “Hiragana”
full	Convert to multi-byte (zenkaku) characters.
mcrypt	MD5 encrypt
half	Convert to single-byte (hankaku) character
scrlen	Outputs the display length of the specified string
sml	Converts roman alphabet letters to lower case.
uconv	UTF-8 <=> Shift JIS / EUC-JP Code Conversion
ugrep	Wrapper for GNU grep
utf8nude	Removes non-displayable ASCII characters and improper UTF8 bytes
width	Returns the display width of a file

SYSTEM

corenum	Outputs the number of physical cores in the server
exist	Checks if one or more files exist
extname	Returns the file extension from a pathname
getip	Returns the IP address of the main machine when you specify the MSCTRLTABLE job group.
gtouch	Creates a null file in gzip format.
ismime	Checks if a file is in MIME formant
itouch	Initializes a file
lcnt	Counts lines (rows)
man2	Display the manual page for a USP command
msctrl	Extracts information from the MSCTRLTABLE
msec	Displays microseconds
rootname	Removes the extension from a path name
semwait	Wait until the specified file exists
ulock	Exclusive access control command

STATISTICAL

abc	ABC (Pareto) Analysis
ceil	Ceiling Function
floor	Floor Function
coefvariation	Coefficient of Variation
correl	Correlation Coefficient (Pearson’s product-moment correlation coefficient)
covariance	Covariance
entropy	Entropy
fft	Fourier Transform
ifft	Inverse Fourier Transform
freqdist	Frequency Distribution
gauss-density	Density Function of the Gauss Distribution
lsfit	Least-Squares Fit
markov-chain	Markov Chain Simulation
mat-calc	Matrix Calculation
mat-minus	Matrix Subtraction
mat-multi	Matrix Multiplication
mat-plus	Matrix Addition
median	Arithmetic Median
mode	Mode
multireg	Multiple Regression
quariledev	Quartile Deviation
range	Range Distribution
sampling	Weighted Sampling of Data
stddev	Standard Deviation
stdscore	Deviation Score
variance	Variance

Usage examples of the most used commands

Command: ctail
Description: deletes the last N rows of a file.

bash> cat example_file
 line1 001
 line2 002
 line3 003
 line4 004

bash> ctail -2 example_file
 line1 001
 line2 002

Command: ccnt
Description: counts the number of fields in each line; it outputs the number of fields of a line each time the previously analysed line has a different number of fields.

bash> cat sample1
 01 02 03
 01 04 05
 01 06 07

bash> cat sample2
 01 A B
 01 C
 01 D
 01 E F G

bash> ccnt sample1
 3

bash> ccnt sample2
 3
 2
 4

Command: self
Description: manipulates the position of the fields (and sections of the fields) in each line.

bash> cat names
 Ernest Rutherford 1871
 Abdus Salam 1926
 Max Planck 1858
 Emmy Noether 1882
 Isaac Newton 1643

bash> self 2 3 1
 Rutherford 1871 Ernest
 Salam 1926 Abdus
 Planck 1858 Max
 Noether 1882 Emmy
 Newton 1643 Isaac

bash> echo one two three four | self 4 3 2 1
 four three two one

bash> echo 02:33 20191217 | self 2.1.4 2.5.2 2.7.2
 2019 12 17

bash> echo A B C D | self NF
 D

bash> echo 01 02 03 04 05 06 | self 1/4
 01 02 03 04

Command: delf
Description: deletes the specified fields in all lines of a file.

bash> cat TABLE
 001 foo bar
 002 bar foo

bash> delf 2 TABLE
 001 bar
 002 foo

bash> delf 2/3 TABLE
 001
 002

Command: sm2
Description: Sums the elements of the specified fields based on the prescribed keys. The first two arguments correspond to the range of fields the key occupies, the last two arguments correspond to the range of fields to be summed.

bash> cat TABLE
 001 foo 1 2
 001 foo 1 2
 002 bar 2 1
 002 bar 2 1

bash> cat TABLE | sm2 1 2 3 4
 001 foo 2 4
 002 bar 4 2

Command: sm4
Description: Sums the elements of the specified fields based on the prescribed keys, displaying subtotals. The first and the second arguments correspond to the range of fields the key occupies, the third and fourth arguments correspond to the range of fields to be ignored and the last two arguments correspond to the fields to be summed.

bash> cat TABLE
 001 foo X 1 100
 001 foo Y 5 200
 001 foo Z 1 100
 002 bar Y 6 400
 002 bar Z 5 300

bash> cat TABLE | sm4 1 2 3 3 4 5
 001 foo X 1 100
 001 foo Y 5 200
 001 foo Z 1 100
 001 foo @ 7 400
 002 bar Y 6 400
 002 bar Z 5 300
 002 bar @ 11 700

Command: sm5
Description: Sums the elements of the specified fields ignoring the key field. The first and the second arguments correspond to the range of ignored key fields, the third and fourth arguments correspond to the fields to be summed.

bash> cat TABLE
 001 foo X 1 100
 001 foo Y 5 200
 001 foo Z 1 100
 002 bar Y 6 400
 002 bar Z 5 300

bash> cat TABLE | sm5 1 3 4 5
 001 foo X 1 100
 001 foo Y 5 200
 001 foo Z 1 100
 002 bar Y 6 400
 002 bar Z 5 300
 @@@ @@@ @ 18 1100

Command: hsum
Description: sums the fields of each line.

bash> cat file1
 1.1 2.2 3.3
 -1 -2 -3

bash> hsum file1
 1.1 2.2 3.3 6.6
 -1 -2 -3 -6

bash> hsum num=1 file1
 1.1 2.2 3.3 5.5
 -1 -2 -3 -5

Command: csum
Description: performs serial addition on a field’s value and outputs the result after each line.

bash> cat TABLE
 foo X 1 3
 foo Y 1 4
 foo Z 1 5
 bar Y 2 8
 bar Z 2 9

bash> csum key=1 val=3/4 TABLE
 foo X 1 1 3 3
 foo Y 1 2 4 7
 foo Z 1 3 5 12
 bar Y 2 2 8 8
 bar Z 2 4 9 17

Command: join0
Description: extracts from the transaction file the lines that, based on a given key, contain items that are described in a given master file.

bash> cat masterF
 1 A 001 foo
 1 A 002 bar
 2 B 002 foofoo
 2 B 003 barbar

bash> cat transactionF
 1 A 001 100
 1 A 002 200
 1 A 003 300
 2 B 001 900
 2 B 002 800
 2 B 003 700
2 B 004 600
 join0 key=1/3 masterF transactionF
 1 A 001 100
 1 A 002 200
 2 B 002 800
 2 B 003 700

Command: cjoin2
Description: associates to each line in a transaction file the master file’s key description; if a key of a given line in the transaction file isn’t present in the master file, the description associated is determined by the user.

cat masterF
 1 A 001 foo
 1 A 002 bar
 2 B 002 foofoo
 2 B 003 barbar

cat transactionF
 1 A 001 100
 1 A 002 200
 1 A 003 300
 2 B 001 900
 2 B 002 800
 2 B 003 700
 2 B 004 600
 join2 +@ key=1/3 masterF transactionF
 1 A 001 foo 100
 1 A 002 bar 200
 1 A 003 @ 300
 2 B 001 @ 900
 2 B 002 foofoo 800
 2 B 003 barbar 700
 2 B 004 @ 600

Command: count
Description: counts the number of lines containing a designated key

bash> cat count_this_file
 001 A iii
 001 A iv
 001 A x
 001 A ix
 002 B vii
 002 B v

bash> count 1 2 count_this_file
 001 A 4
 002 B 2

Command: map
Description: converts a list of lines where keys are repeated into a table where the repeated keys are intertwined in such a way that the information is more readable.

bash> cat sample
 A 01 40
 A 02 50
 A 03 60
 B 01 70
 B 02 80

bash> map -m@ num=1 sample
 * 01 02 03
 A 40 50 60
 B 70 80 @

Command: transpose
Description: transposes tables.

bash> cat sample
 A 01 40
 A 02 50
 A 03 60
 B 01 70
 B 02 80

bash> transpose sample
 A A A B B
 01 02 03 01 02
 40 50 60 70 80

Command: keycut
Description: divides files according to the keys.

bash> ls
 LIST

bash> cat LIST
 A 001 foo
 A 002 bar
 B 001 foofoo
 B 002 barbar

bash> keycut CLASS.%1 LIST

bash> ls
 LIST CLASS.A CLASS.B
bash> cat CLASS.A
 A 001 foo
 A 002 bar

bash> cat CLASS.B
 B 001 foofoo
 B 002 barbar

Find out more

Request a demo and speak with our team about how you can leverage the power of Unicage in your organization.