- 1 Overview
- 2 Installation
- 3 Functions and Macros
- 4 Special Variables
A collection of eclectic, battle-tested Common Lisp utilities for everyday programming tasks.
dc-eclectic provides a grab-bag of high-quality utility functions for file operations, string manipulation, hashing, random generation, list processing, path handling, and more. These functions are designed to be practical, performant, and easy to use.
ros install macnod/dc-eclectic/{release}where {release} looks like vX.XX. You can find the latest release in the repo.
Clone the repo to a directory that is visible to ASDF, such as ~/common-lisp, then, in a REPL, do (ql:quickload :dc-eclectic).
Alphabetical list of functions and macros that DC-UTILITIES defines.
-
[function] ALL-PERMUTATIONS LIST
Returns a list of every permutation of elements in
LIST. For example: '(1 2 3) -> '((1 2 3) (1 3 2) (2 1 3) (2 3 1) (3 1 2) (3 2 1))
-
[function] ALL-PERMUTATIONS-OF-STRING S
Returns a list of strings representing every permutation of the orginal string
S. For example: "abc" -> '("abc" "acb" "bac" "bca" "cab" "cba")
-
[function] ASCII-ALPHA
Return a string with all the alphabetic ASCII characters.
-
[function] ASCII-ALPHA-LOWER
Returns a string with all the lower-case alphabetic ASCII characters (a-z).
-
[function] ASCII-ALPHA-NUM
Returns a string with all the alpha-numeric ASCII characters.
-
[function] ASCII-ALPHA-NUM-LOWER
Returns a string with all the lower-case alpha-numeric ASCII characters.
-
[function] ASCII-ALPHA-NUM-UPPER
Returns a string with all the upper-case alpha-numeric ASCII characters.
-
[function] ASCII-ALPHA-UPPER
Returns a string with all the upper-case alphabetic ASCII characters (A-Z).
-
[function] ASCII-CHAR-RANGE BEGIN END
Returns a string that includes ASCII characters in the ASCII code range given by
BEGINandEND.
-
[function] ASCII-NUMERIC
Returns a string with all the ASCII characters that represent digits.
-
[function] BASE64-DECODE BASE64-ENCODED-STRING
Decodes
BASE64-ENCODED-STRINGand returns the result. This function is compatible with base64-encode and with popular implementation of base64 encoding and decoding functions. However, this function doesn't properly handle multi-byte characters. Unless you require compatibility with external functions, use the other decoding functions inDC-ECLECTIC. SeeSAFE-ENCODE,SAFE-DECODE, andDEFINE-BASE-ENCODER.
-
[function] BASE64-ENCODE STRING
Returns a base64-encoded version of
STRINGthat that works list like the standard base64-encode implementations in the Linux command line and Emacs Lisp. These do not properly handle multibyte characters, so use this function only if you need compatibility with popular implementations.DC-ECLECTICcontains better encoding and decoding functions. SeeSAFE-ENCODE,SAFE-DECODE, andDEFINE-BASE-ENCODER.
-
[function] CHOOSE-ONE SEQ &OPTIONAL RSTATE
Returns a random element from
SEQ.
-
[function] CHOOSE-SOME SEQ N &OPTIONAL RSTATE
Choose
Nelements fromSEQand return a new sequence with those elements, with the new sequence having the same type asSEQ.Nmust be an integer that is greater than or equal to 1.If
Nis greater than the length ofSEQ, this function returns a copy ofSEQ.If
Nis less than 1, then this function returns nil, even if the sequence is not a list.This function uses the
RANDOMfunction, which accepts an optionalRSTATEparameter that you can optionally pass in here.
-
[function] COMPARABLE-HASH-DUMP HASH &KEY (F-SORT #'STRING<) (F-MAKE-SORTABLE (LAMBDA (K) (FORMAT NIL "~a" K))) FLAT
Turns the given hash table into a list of pairs or, if
FLATisT, into a plist that represents the hash. In the resulting list, the keys are sorted, so that the list can be more easily compared with other lists.HASHis the hash you want to dump.F-SORTis the sort predicate, which defaults to sorting strings in ascending order.F-MAKE-SORTABLEis a function that accepts a key and returns a sortable version of the key. This defaults to turning the key into a string. If the keys to the hash are integers, for example, you can provide anF-SORTof #'<, which sorts integers numerically in ascending order, and you can provide anF-MAKE-SORTABLEof #'identity, which will leave the keys as integers for sorting purposes.
-
[function] COPY-FILE SOURCE DESTINATION &KEY (IF-EXISTS :SUPERSEDE) (BUFFER-SIZE (* 64 1024))
Copies
SOURCEfile toDESTINATIONfile. IfDESTINATION's directories do not exist, they are created.IF-EXISTScontrols the behavior ifDESTINATIONalready exists, and may be :error, :new-version, :rename, :rename-and-delete, :overwrite, :append, or :supersede.IF-EXISTSdefaults to :supersede.BUFFER-SIZEcontrols the size of the buffer used during the copy operation, and defaults to 64 KB. Returns theDESTINATIONpath.
-
[function] DEEP-COPY THING
Deep copy
THING, recursively copying lists, vectors, and strings.
-
[macro] DEFINE-BASE-ENCODER NAME ALPHABET
Creates 2 functions using
NAME(a symbol), andALPHABET(a string): {NAME}-ENCODE and {NAME-DECODE}. {NAME}-ENCODE takes 1 parameter,DATA(a string), and returns the string encoded withALPHABET. {NAME}-DECODE accepts a single parameter (an encoded string) and returns the original strinng. For example, if you need to encode strings using hexadecimal characters, you can call this function as follows:(define-base-encoder hex "0123456789abcdef")That will produce the functions HEX-ENCODE and HEX-DECODE. Those function will work as follows:
(hex-encode "Hello World!") ;; => "48656c6c6f20576f726c6421" (hex-decode "48656c6c6f20576f726c6421") ;; => "Hello World!"
-
[function] DENORMALIZE-LIST LIST MIN MAX &KEY INTEGER
Returns a new list with the numbers in
LIST, which are floating-point numbers between 0 and 1, expanded to the rangeMAX-MIN, such that the number 1.0 is converted toMAX, the number 0.0 is converted toMIN, and all the other numbers fall in the rangeMINtoMAX.INTEGERisT, the new list contains integers. Otherwise, the new list contains floating-point numbers.
-
[function] DIRECTORY-EXISTS-P PATH
Returns a boolean value indicating if the directory specified by
PATHexists.
-
[function] DISTINCT-ELEMENTS SEQUENCE &KEY (KEY #'IDENTITY)
Accepts a sequence of elements (list or vector) and returns a new sequence of the same type with distinct elements from the original. If the elements in the sequence are hash tables, plists, or objects with methods, then you can provide a value or function for the :key parameter. If you provide a value, the function will use the value as the key of the element, and the value of the key will represent the unique signature of the element. If you provide a function, then the function will be applied to the element to compute the elements unique signature.
-
[function] DISTINCT-STRINGS LIST
Alternative for returning a list of distinct strings.
-
[function] DISTINCT-VALUES LIST
Synonym for
DISTINCT-ELEMENTS.
-
[function] ENDS-WITH S SUFFIX
Returns
Tif stringSends withSUFFIX.
-
[generic-function] EXCLUDE REFERENCE-LIST EXCLUDE
Returns a list containing the elements of
REFERENCE-LISTthat are not inEXCLUDE. IfEXCLUDEis a list, this function excludes all elements inEXCLUDEfromREFERENCE-LIST. IfEXCLUDEis a string, this function excludes the string fromREFERENCE-LIST
-
[function] EXCLUDE-REGEX REFERENCE-LIST EXCLUDE &OPTIONAL EXCEPTIONS
Returns a list of the elements of
REFERENCE-LISTthat that don't match theEXCLUDEregular expression. However, elements that are not inEXCEPTIONSare not excluded, even if they matchEXCLUDE.
-
[function] EXISTING-N-GRAM-STRINGS CHARS COUNT HASH
Returns all the
COUNTn-grams for stringCHARS, but only of n-grams that exist as keys inHASH.
-
[function] EXISTING-PERMUTATIONS-OF-STRING S HASH
Works just like all-permutations-of-string, but excludes any permutations of
Sthat are not among the keys inHASH.
-
[function] FILE-EXISTS-P PATH
Returns a boolean value indicating if the file specified by
PATHexists.
-
[function] FILE-EXTENSION PATH
Returns a string consisting of the file extension for the file name given in
PATH.
-
[function] FILENAME-ONLY FILENAME
Retrieves the filename (filename only, without the path) of
FILENAME.
-
[function] FLATTEN L
Given a nested list
L, return a flat list. If an array or other sequence is among the elements ofL, the sequence is not flattened, but treated as a single element.
-
[function] FREEZE OBJECT
Stringifies
OBJECT.
-
[function] GETENV NAME &KEY DEFAULT REQUIRED (TYPE :STRING)
Get the value of the environment variable
NAME, returning a string or an integer, depending onTYPE. If the environment variable is not set, then returnDEFAULT, which must be of typeTYPE. If the environment variable is not set andDEFAULTisNIL, then this function returnsNIL.TYPEcan be :integer, :string, or :boolean.
-
[generic-function] HAS REFERENCE-LIST THING
Returns
TifREFERENCE-LISTcontainsTHING. IfTHINGis a string, number, or symbol (including keywords), this function checks for that item inREFERENCE-LIST. IfTHINGis a list, this function checks that all elements ofTHINGare inREFERENCE-LIST.
-
[function] HAS-SOME REFERENCE-LIST QUERY-LIST
Returns
TifREFERENCE-LISTcontains any of the elements inQUERY-LIST.
-
[function] HASH-HMAC-256 SECRET TEXT
Hash
TEXTusingSECRETand hmac-sha-256 and return a hex representation of the hash
-
[function] HASH-KEYS HASH
Returns a list of the keys in hash table
HASH.
-
[function] HASH-STRING STRING &KEY (SALT "") (SIZE 128)
Hash
STRINGand return a hex representation of the hash
-
[function] HASH-VALUES HASH
Returns a list of the values in hash table
HASH.
-
[function] HASHIFY-LIST LIST &KEY (METHOD :COUNT) F-KEY HASH-KEY PLIST-KEY ALIST-KEY (F-VALUE (LAMBDA (KEY-RAW KEY-CLEAN VALUE) (DECLARE (IGNORE KEY-RAW KEY-CLEAN)) VALUE)) (INITIAL-VALUE 0)
Creates a hash table from
LISTand returns the hash table, according toMETHOD. Supported methods are:COUNT,:PLIST,:ALIST,:INDEX,AND(01):CUSTOM.:COUNTWith the :COUNT method, which the function uses by default, the function creates a hash table in which each key is an item of the list and the associated value for each key is the incidence of the item in the list. For example: (hashify-list '(7 8 7 7 8 9)) gives you a hash table that looks like this: {7: 3, 8: 2, 9: 1}:ALISTand:PLISTThe :ALIST and :PLIST methods convert the list into a hash that conceptually represent the same map as the list. Alists and plists both consist of collections of key/value pairs. Alists look like this: '((key1 . value1) (key2 . value2) (key3 . value3)...) Plists look like this: '(:key1 value1 :key2 value2 :key3 value3 ...) If a key repeats in one of these lists, its value simply overwrites the value of the repeated key. However, you can change that behavior. See the description of the :CUSTOM method for information on how to do that.:INDEXThe :index method causes the values in the list to become the keys in the hash table. The value associated with each key should be an increasing integer, starting with 0. Thus, the list '(a b c) becomes the hash {a: 1, b: 2, c: 3}. If the objects in the list that you're indexing are hash tables, then you can specify the object key for the value that the function should use as a key in the resulting hash. That object key should be present in every object in the list. This allows you to index a list of hash tables by some specific value in the hash table. Consider the following example: [ {id: "a-001", first: "john", last: "doe"}, {id: "a-002", first: "jane", last: "doe"} ] If you specify :method :index :hash-key "id", this function will create a hash table that looks like this: { "a-001": {id: "a-001", first: "john", last: "doe"}, "a-002": {id: "a-002", first: "jane", last: "doe"} } And, voilá, you no longer need to iterate through a list to find your object. If the objects are plists, and you specify the index with plist-key, you'll see the same behavior with the plist as we demonstrated above for hash tables. HASH-KEY and PLIST-KEY are just shortcuts to save you from having to write some code for F-KEY. You can specify only one of HASH-KEY, PLIST-KEY, and F-KEY.:CUSTOMThe :CUSTOM method requires that you provide functions for computing the keys and values that the function inserts into the resulting hash. Use F-KEY to provide a function that accepts an element from LIST and returns a computed hash key. Here are some examples F-KEY of acceptable definitions: - #'identity - #'string-upcase - (lambda (x) (zerop (mod x 10))) Use F-VALUE to provide a function that accepts an element from LIST, the computed key (which might be different from the element), and the value that's currently associated with the computed key in the resulting hash table. Here are some examples: - (lambda (element computed-key value) (declare (ignore element computed-key)) value) - (lambda (element computed-key value) (declare (ignore element value)) (incf value)) If there's no hash value associated with the computed key, then the value specified by :INITIAL-VALUE is used.
-
[generic-function] INDEX-OF-MAX LIST-OR-VECTOR
Returns the index of the largest number in
LIST-OR-VECTOR.
-
[function] JOIN-PATHS &REST PATH-PARTS
Joins parameters (collected in
PATH-PARTS) into a unix-like file path, inserting slashes where necessary.PATH-PARTScan be strings or pathnames. If the first element inPATH-PARTSstarts with a slash, the resulting path will be absolute; otherwise, it will be relative. If the last element ofPATH-PARTSends in a slash, the resulting path will also end in a slash.PATH-PARTSelements should be strings or pathnames, but this function will try to convert other types to strings, if possible.NILor empty strings are ignored.
-
[function] MAKE-KEYWORD S
Returns a keyword form of the string
S. Examples:hello world => :HELLO-WORLD -hello--world--- => :HELLO-WORLD hello__world => :HELLO-WORLD
-
[function] N-GRAM-STRINGS CHARS COUNT
Accepts
CHARS, a string, andCOUNT, an integer, and returns all the possible combinations of lengthCOUNTof the characters inCHARS. For example, (n-gram-strings "abc" 2) => '("aa" "ab" "ac" "ba" "bb" "bc" "ca" "cb" "cc")
-
[function] N-GRAMS LIST COUNT
Returns n-gram lists of length
COUNTusing elements ofLIST.
-
[function] NORMALIZE-LIST LIST &KEY MAX MIN
Returns a new list with new values between 0.0 and 1.0.
MAXis the largest value thatLISTcan hold, andMINis the smallest. Each new value N is computed from the corresponding old value O inLIST, as follows:N = (O -
MIN) / (MAX-MIN)If you don't provide
MAXandMIN, this function does an initial pass through list where it setsMAXandMINto the largest number and the smallest number inLIST, respectively. Therefore, you can improve the performance of this function if you already know those values. Furthermore, in some cases the list may not even contain the values forMAXandMINthat you need.
-
[function] PATH-LEAF-ONLY PATH
Returns the last part of the directory
PATH. For example, /home/one/two => two. IfPATHis /, this function returns /.
-
[function] PATH-ONLY FILENAME
Retrieves the path (path only, without the filename) of
FILENAME.FILENAMEshould be a string, a pathname, orNIL. IfFILENAMEisNILor the empty string, this function returns the empty string. IfFILENAMEhas no path component, this function returns "/".
-
[function] PATH-PARENT PATH
Returns the absolute path to the directory that is the parent of
PATH.PATHmust be an absolute path. IfPATHpoints to a file, then this returns the path to the file, minus the file name. IfPATHpoints to a directory, this returns the parent of the directory inPATH. IfPATHpoints to a directory, it must end in /. IfPATHis / or not an absolute path, this function returnsNIL.
-
[function] PATH-TYPE PATH
Returns
:FILE,:DIRECTORY, or:NOT-FOUND, depending on whatPATHpoints to.
-
[function] PLIST-KEYS PLIST
Returns a list of the keys in
PLIST.
-
[function] PLIST-VALUES PLIST
Returns a list of the values in
PLIST.
-
[function] PLISTP THING
Returns
TifTHINGis a plist.
-
[function] PLURAL WORD
Convert singular to plural form (works most of the time). Return value is always in lower case.
-
[function] RAND VALUE &OPTIONAL RSTATE
Returns a random number below
VALUE. WithRSTATE, draws from that random state. Without it, draws from default-rstate, which is lazily seeded from the OS on first use (per-process randomness; not the global random-state).
-
[function] RANDOM-HEX-NUMBER &OPTIONAL (DIGITS 7) (NON-ZERO-START) RSTATE
Returns a random hexadecimal number with
DIGITSdigits. IfNON-ZERO-STARTis specified, then the resulting hexadecimal number starts with a character other than 0.
-
[function] RANDOM-NUMBER &OPTIONAL (DIGITS 4) RSTATE
Returns a random integer of
DIGITSdigits.
-
[function] RANDOM-STRING STRING-LENGTH ALPHABET &OPTIONAL RSTATE
Returns a random string of length
STRING-LENGTH. The string is constructed from characters inALPHABET.ALPHABETis a string. There are several functions available for buildingALPHABET, all starting with the prefix 'ASCII-'. For example:(random-string 10 (ascii-alpha-num-lower))
-
[function] RANGE START END &KEY (STEP 1) (FILTER #'IDENTITY) SHUFFLE
Returns a list of values between
STARTandEND(inclusive), skipping values bySTEP, filtering remaining values with the function inFILTER, and shuffling the remaining values ifSHUFFLEis true.STEPdefaults to 1,FILTERdefaults to allowing all values through, andSHUFFLEdefault to nil.
-
[function] REPLACE-EXTENSION FILENAME NEW-EXTENSION
Replaces the file extension in
FILENAMEwith the file extension provided inNEW-EXTENSION.
-
[function] ROOT-PATH FILES
Given
FILES, a list of paths in the form of strings, returns the starting path that all the paths have in common.
-
[function] SAFE-DECODE DATA
Decodes
DATAusing theSAFEalphabet.
-
[function] SAFE-ENCODE DATA
Encodes data using the
SAFEalphabet.
-
[function] SAFE-SORT LIST &KEY PREDICATE
Returns a sorted copy of
LIST, without modifyingLIST. IfPREDICATEis not provided, the function determines the type of the first element of the list to pick a suitable sort predicate. The function assumes that all the elements inLISTare of the same type and that they are either strings, keywords, or numbers. The function sorts in ascending order by default. IfPREDICATEis provided, then the function uses that predicate.
-
[function] SETENV NAME VALUE
Set environment variable
NAMEtoVALUE.VALUEis always converted into a string. Returns a string withVALUE.
-
[function] SHELL-COMMAND-BACKGROUND COMMAND &KEY (WAIT-INTERVAL 0.1)
Run
COMMANDin the background. Returns a process-info plist with keys: :pid, :exit-code (nil if running), :output, :error-output, :running-p, :wait-interval (for polling), :status.Returns an
INFOobject that you can use later to check on the backgrounded process. Use (SHELL-COMMAND-RUNNING-PINFO), (SHELL-COMMAND-WAITINFO), (SHELL-COMMAND-EXIT-CODEINFO) to check status.
-
[function] SHELL-COMMAND-RUNNING-P INFO
Returns
Tif the background process is still running.INFOis the object thatSHELL-COMMAND-BACKGROUNDreturns when it starts the process.
-
[function] SHELL-COMMAND-TO-STRING COMMAND
Executes
COMMANDin the shell and returns the output as a string.
-
[function] SHELL-COMMAND-WAIT INFO &OPTIONAL TIMEOUT
Wait for process to finish (with optional
TIMEOUTseconds).INFOis the object thatSHELL-COMMAND-BACKGROUNDreturns when UpdatesINFOwith output and exit code. Returns updatedINFO. IfTIMEOUTis ommitted orNIL, waits until the process finishes, which might be forever if the process is a server process, for example. IfTIMEOUTis provided and the process does not finish withinTIMEOUTseconds, the process is terminated. When the process terminates normally,INFOis updated with :status :completed, the exit code, and the output and error-output strings. When the process is terminated due a timeout,INFOis updated with :status :terminated, and the exit code, but output and error-output are leftNIL.
-
[function] SHUFFLE SEQ &OPTIONAL RSTATE
Return a sequence with the same elements as the given sequence S, but in random order (shuffled).
-
[function] SINGULAR WORD
Convert plural to singular form (works most of the time). Return value is always in lower case.
-
[function] SLURP FILENAME
Returns a string with the content of the file at
FILENAME.
-
[function] SPEW STRING FILENAME
Write
STRINGto filename, creating the file if necessary, and replacing the contents of the file if the file already exists.
-
[function] SPLIT-N-TRIM STRING &KEY (ON-REGEX "\\s+") (FAT "^\\s+|\\s+$")
Splits
STRINGinto substrings onON-REGEX, then trimsFATfrom each substring. TheON-REGEXparameter value, which is optional, defaults to "\s+", which is to say that the string is split into a list of words at the whitespace boundaries. The default value forFAT, which is also optional, "\s+|\s+$", causes this function to trim whitespace from the beggining and end of each substring. Here's an example:(split-n-trim "Hello beautiful world!") => '("Hello" "beautiful" "world!")
-
[function] STARTS-WITH S PREFIX
Returns
Tif stringSstarts withPREFIX.
-
[function] THAW STRING
Returns an object resulting from the evalation of
STRING. This is the opposite ofFREEZE.
-
[function] TO-ASCII STRING &KEY (REPLACEMENT-CHAR #\?) (PRINTABLE-ONLY T)
In
STRING, replaces non-ASCII characters with REPLACEMENT_CHAR, which defaults to the question mark. IfPRINTABLE-ONLYis true, only printable ASCII characters are kept, with the rest being replaced byREPLACEMENT-CHAR.
-
[function] TREE-COL TREE &REST PATH
Treat all the elements in
PATHminus the last element as the node path and treats the last element ofPATHas the column key. The node path should lead to a node that consists of a list of plists. This list can beNIL. This function locates the node given by the node path and retrieves the values associated with the column key from the plists in the node, returning those values ina list. For example, ifPATHis '(:a :b :c), and the node at path '(:a :b) is a list of plists, this function returns a list consiting of the value associated with :c in every plist at '(:a :b). If the node at the node path does not exist, this function returns nil. If a node at the node path exists and is not a plist (nil counts as a plist), then this function signals an error.
-
[function] TREE-GET TREE &REST PATH
Get value from the
TREEstructure, atPATH.TREEis a nested data structure where each value can be a plist, list, object, t, or nil.
-
[macro] TREE-PUT VALUE TREE &REST PATH
Set
VALUEat the location specified byPATHin theTREEstructure. Expands into a series ofgetfandnthcalls for efficient access.
-
[function] TRIM S &OPTIONAL (FAT "^\\s+|\\s+$")
Trim
FATfrom the string inS. TheFATparameter is optional and defaults to "^\s+|\s+$", which means "Whitespace at the beginning or end of the string".
-
[function] TRIM-WHITESPACE S
Trim Whitespace from the beginning and end of
S.
-
[function] UUID &OPTIONAL RSTATE
Returns a random
UUIDstring.
-
[function] VERIFY-STRING STRING REGEX &KEY IGNORE-CASE
Return t if
STRINGmatches theREGEXexactly. Use theIGNORE-CASEparameter if you want case-insensitve matches.
-
[function] ZIP &REST LISTS
Returns a new list with elements from the multiple lists in
LISTSmerged. Each list inLISTScan be of any size. This function picks the first element from each list and adds it to the result, then picks the second element of each list, and adds it to the result, and so on. If any list runs out of elements, the function continues performing the same operation with the lists that still have elements left. This function does not alter the original lists. Examples:(zip '(1 4 7) '(2 5 8) '(3 6 9 10)) ;; => '(1 2 3 4 5 6 7 8 9 10)
Exported special variables.
-
[variable] *ALPHABET-ALPHANUM* "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
Alphabet with all ASCII letters and numbers.
-
[variable] *ALPHABET-BITCOIN* "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
Bitcoin alphabet. No zero, no capital 'o' or 'i', and now lowercase 'l'.
-
[variable] *ALPHABET-ALPHANUM-UPPER* "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
Alphabet with uppercase letters and numbers.
-
[variable] *DEFAULT-RSTATE* NIL
Shared random state for the random generators and uuid when no
RSTATEis passed. Lazily seeded from the OS on first use; nil until then. Bind to (reference-random-state) for deterministic output.
